> That kind of stuff is necessary for binary protocols to evolve in a compatible way. When a TLS 1.4 is defined, we need a window where clients and servers can still negotiate 1.3 until both have been upgraded. And 1.3 had to find ways to be compatible with 1.2, and so forth. Decades of that kind of evolution are guaranteed to leave some marks in the protocol.
It's necessary because people are incompetent, and because overall the market rewards them for incompetence. From the outset TLS provided a trivial version negotiation mechanism but it was easier to ignore it and write incompatible garbage especially for so-called "Middle boxes" often sold as a drop-in security "solution" for businesses.
So when it came time to ship TLS 1.1, it was soon discovered that in practice you can't just say "Hi, I speak TLS 1.1" which would be a couple of bytes - that won't work, you need to find some other way to quietly signal to competent people that you know the newer protocol. So they did, slightly weakening the security in the process, and this continued into TLS 1.2 where browsers began doing "Fallback" which was a risky but sadly necessary process where you give up attempting the new protocol altogether sometimes, thus opening yourself up to supposedly obsolete attacks.
By TLS 1.3 things had become so bad that TLS 1.3 essentially begins, as you can see if you inspect the data shown on that page, by pretending we're speaking TLS 1.2 and then saying we want to negotiate an optional "extension" to TLS 1.2 which is where we confess we actually speak TLS 1.3.
Every single packet of TLS 1.3 encrypted data is also wrapped in a TLS 1.2 layer saying "Don't mind me, I'm just application data". Why? Because we can't confess we're not speaking TLS 1.2, ever, and if we said we were doing TLS 1.2 crypto system stuff the same incompetent garbage software would try to get involved because it "understands" (badly) how to speak TLS 1.2, so we just pretend it missed the negotiation phase, this is just application data, nothing to see.
And it works. That's crucial. It's why we did all this, and yet it reveals that because the products people bought were developed incompetently they wouldn't even have detected serious attacks anyway, let alone prevented them. Need to sneak 40GB of stolen financial data over a network "protected" by this Genuine Marketing Leading Brand Next Generation Firewall? Don't worry, just label it "application data" with no explanation and it'll be completely ignored.
If you've ever watched a Lock Picking Lawyer video on Youtube, it was like one of the ones where it's several minutes so you expect it'll be hard to pick, but then you find out he's actually so disgusted by the lacklustre security of this $150 "Pick Proof High Security Lock" that although he rakes it open in 2 seconds with a cheap tool, and then shims it open with a discarded Redbull can, and then knocks it open with a hammer, and then uses a purpose built bypass tool to open it instantly in a single flowing motion, he also takes time to disassemble it and show you that the manufacturer fucked up, wasting material solving a non-existent problem and in the process making the lock much worse, which is why the video was so long.
Learning from their experience with these "Security" products for TLS 1.3, the QUIC people designed QUIC specifically with the intent that you can't even tell what version it is unless you're the client or the server, and then they shipped a new QUIC version to check that works even though they don't really need one yet, so that they don't have to do this whole dance again every few years.
While I mostly agree, I wouldn’t attribute buying middleboxes (or building shitty middleboxes) to incompetence. This is about doing the minimum viable effort for compliance, not about actual security, and everyone who bought or sold these boxes knew that.
In my opinion we should consider doing things just for compliance without any real benefit as incompetence. I know this isn’t the reality we live in at the moment however. Way too many things are just box-checking and I find that frustrating.
From what I understand of the parent's comment, there wasn't even the minimum viable effort for compliance, because the version checking system doesn't comply with the TLS standard.
I think _hl_ is thinking of compliance in terms of compliance with a government regulation, or internal company policy, rather than compliance with the TLS standard.
Large organisations may even have a department named compliance whose job is to ensure the whole company obey certain policies, or at least in most cases, that they avoid being fined / prosecuted / shut down for not complying.
In practice this shades over, Compliance may include the 3rd party contractor who wanders into a C suite meeting, wearing somebody else's badge to point out that er, company security policy isn't being obeyed, you there, Bob, this badge I'm wearing is your badge, why are you in this meeting without your badge and why is your badge, now around my neck, still working even though you lost it last week?
But it will also include Sarah, who would really rather be playing Solitaire, and is implementing the policy that everybody needs to check this box which says "I have no outside interests in violation of Federal Rule 1234.567". What is Federal Rule 1234.567 ? Sarah doesn't know and doesn't care. She also has no clue what you should do if you in fact do have outside interests which violate this rule, she just wants you to check the box.
When it turns out the company has over 800 staff in violation of 1234.567 the lawyers will insist "But we checked none of them were violating the rule" and so the company didn't break the rules.
People like Sarah caused the shitty systems to be installed that, you're right, do not comply with the TLS specification, but it's not really Sarah's fault, people like her are always going to exist, we should make it harder to get it wrong than not to, so that laziness results in success.
I agree it's probably not Sarah's fault, but certainly the company would be at fault for incorrectly implementing the TLS standard. Especially if they advertised TLS support to their customers and users.
It's necessary because people are incompetent, and because overall the market rewards them for incompetence. From the outset TLS provided a trivial version negotiation mechanism but it was easier to ignore it and write incompatible garbage especially for so-called "Middle boxes" often sold as a drop-in security "solution" for businesses.
So when it came time to ship TLS 1.1, it was soon discovered that in practice you can't just say "Hi, I speak TLS 1.1" which would be a couple of bytes - that won't work, you need to find some other way to quietly signal to competent people that you know the newer protocol. So they did, slightly weakening the security in the process, and this continued into TLS 1.2 where browsers began doing "Fallback" which was a risky but sadly necessary process where you give up attempting the new protocol altogether sometimes, thus opening yourself up to supposedly obsolete attacks.
By TLS 1.3 things had become so bad that TLS 1.3 essentially begins, as you can see if you inspect the data shown on that page, by pretending we're speaking TLS 1.2 and then saying we want to negotiate an optional "extension" to TLS 1.2 which is where we confess we actually speak TLS 1.3.
Every single packet of TLS 1.3 encrypted data is also wrapped in a TLS 1.2 layer saying "Don't mind me, I'm just application data". Why? Because we can't confess we're not speaking TLS 1.2, ever, and if we said we were doing TLS 1.2 crypto system stuff the same incompetent garbage software would try to get involved because it "understands" (badly) how to speak TLS 1.2, so we just pretend it missed the negotiation phase, this is just application data, nothing to see.
And it works. That's crucial. It's why we did all this, and yet it reveals that because the products people bought were developed incompetently they wouldn't even have detected serious attacks anyway, let alone prevented them. Need to sneak 40GB of stolen financial data over a network "protected" by this Genuine Marketing Leading Brand Next Generation Firewall? Don't worry, just label it "application data" with no explanation and it'll be completely ignored.
If you've ever watched a Lock Picking Lawyer video on Youtube, it was like one of the ones where it's several minutes so you expect it'll be hard to pick, but then you find out he's actually so disgusted by the lacklustre security of this $150 "Pick Proof High Security Lock" that although he rakes it open in 2 seconds with a cheap tool, and then shims it open with a discarded Redbull can, and then knocks it open with a hammer, and then uses a purpose built bypass tool to open it instantly in a single flowing motion, he also takes time to disassemble it and show you that the manufacturer fucked up, wasting material solving a non-existent problem and in the process making the lock much worse, which is why the video was so long.
Learning from their experience with these "Security" products for TLS 1.3, the QUIC people designed QUIC specifically with the intent that you can't even tell what version it is unless you're the client or the server, and then they shipped a new QUIC version to check that works even though they don't really need one yet, so that they don't have to do this whole dance again every few years.