It is truly surprising how large the specs for each of these ecosystems are: PKI, Kerberos, TLS, OAuth, SAML, etc. They are gargantuan, especially when you include essential dependencies like DER codecs and ASN.1 compilers (PKI, Kerberos) or XML (SAML).
“There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult.”
The problem with this is the assumption that there's a simpler way to do all of this, and if only we could stop looking for large and complex solutions we could just land on the simple ones.
I listed a number of solutions, all built by different people, at different times, in different orgs, and some of those solutions (OAuth, SASL) being much more organic in how they evolved, and yet all are ultimately large and complex.
I think that hints at the problem space being... large and complex and requiring large and complex solutions.
What we _can_ do is avoid adding complexity unnecessarily, but what looks like a simplification today (e.g., picking the best current encoding system) might look like a terrible mistake in twenty years.
The thing is that they don't have to be that big at all, you could probably specify enough of PKI and TLS and SSH to cover most uses cases in, I dunno, 30-40 pages. However the standards bodies that produced them, termed "working groups", are more like standing committees that will (a) standardize any random idea that any member brings along and (b) are worse than the energizer bunny, they just keep going and going and going and going. Even ones that have been forcibly shut down like PKIX just keep going in other forms (LAMPS). You can't stop these standards mills, they'll just keep grinding out more stuff that no-one ever asked for, for all of eternity.
It sure sounds like it should be like this, but when you actually try you end up with not this. TLS is huge! Yes, but SSL 2.0 was smaller, and buggy as hell, so it had to evolve, and after 30+ years it became the monster that it is today.
Of course, SSL 2.0 did reference x.509, so hey, SSL 2.0 should have invented its own PKI. Except that Netscape might have come up with something terrible that worked in 1993 in labs but didn't scale to the web, or just full of security problems, or...
What you say sounds nice and right right up until you actually look at the details of what actually happened in real life, and how things actually evolve when they have little standards involvement.
Well, no-one held a gun to their head but there was a general "enough, already", aided by the fact that several of the main characters were retiring which helped wind it up. And then they just kept going as before under a new name and with an influx of new people who were unaware of how the original mess was made and why.
So if a WG is wound up and then continues under another name with mostly the same people doing the same things the original WG did it's not really wound up, is it? It's just changing the sign over the door with business continuing as usual.
First, anyone can participate. The only cost is the value of your time.
Second, yes, there are the usual suspects -- the ones who've decided to spend a lot of their time on whatever the area of tech we're talking about.
Third, working groups have charters that delineate what RFCs they will publish. Sometimes the work runs out. Sometimes the people run out of energy. Sometimes the tech is 'done', at least for a while. Then the WGs shut down.
Fourth, sometimes new work gets brought to the IETF in an area where the relevant WG has concluded, so then a new WG _may_ get spun up to take on that work.
Re: canonicalization: design things to not need it, so don't bother with it. Relying parties need to received the blob of whatever (XML, JSON, DER, PB -- don't care or decode till the signature is validated), validate the signature over the exact blob you've received, then decode. This means you need the signing key's algorithm to be identifiable from its issuer and key ID metadata / URI. The header you'll need should only have the information you need to find (dereference) the issuer's signing public key. You'll never need to canonicalize anything this way.
Ideally you should use encrypted tokens so you're forced to do authenticated decryption before getting to the claims portion, but no one has bothered to build that in a way that scales. I did build something like this for Kerberos in Heimdal (https://github.com/heimdal/heimdal), but we need it for encrypted JWTs too: derive symmetric encryption keys from {current epoch, base key, audience/relying party name}, and let the relying party download those keys (by authenticating) much the way they do for signed JWTs. But switching from signed JWTs to encrypted ones is a pain as it requires that the issuer know if the relying party can handle it.
Right. Watermarks are the same in every copy -- they mark a copyright or whatever. Whereas these things are personalized, therefore they do track you as a source of sharing.
Sun's various divisions all cared very much about their respective vendor lock-in, so they didn't innovate fast enough. The divisions that didn't have that innovated. On the way downward Sun cannibalized itself (think Sun PS getting closed).
I've written several times here about all the mistakes that Sun made during the 00s that doomed it. Among the many mistakes it made are:
- Cancelling -if briefly- Solaris on x86 in 2002. This killed Solaris in the minds of many who didn't want to be locked into Sun for SPARC.
- Failing to make a deal with Google in 2002. Apparently Sun insisted on knowing how many servers Google had, something that Google considered a high-value secret, so Sun failed to make a deal with Google, so Google ended up using Linux and contributing to Linux. This was a tremendous mind-share disaster -- it's hard to overestimate the damage done by this.
- Closing Sun PS (professional services). Bad bad move, possibly the worst of them.
- Not giving up on J2ME earlier -- it's not the sort of thing that could last forever, and Steve Jobs killed it with the iPhone. This was a case of vendor lock-in clouding Sun's decision making.
- Failure to recognize that Sun needed to become a systems company, not a CPU company.
- Failure to respond to Active Directory. This was yet another case of vendor lock-in clouding Sun's decision making: the Sun DS product team was milking their existing customers more than they wanted to go after more business with a sustainable strategy.
- UltraSPARC was more than a decade too late to make up for SPARC falling way behind x86_64. Sun needed to give up on SPARC, but again, vendor lock-in sounds sweet but turns out to be poison.
- Failure to make a deal with Apple for it to use ZFS in OS X.
- The MySQL purchase. WTF, this was horrible and stupid. The only interesting effect of this was to make Sun a target of acquisition for Oracle. But of course, it turns out that Oracle -a company built on building mind-share- had become too blinded by vendor lock-in just like Sun, so...
There were numerous other mistakes along the way. These are the most salient, for me anyway.
What's shocking is how long it took Sun to fail under those circumstances!
Also shocking is how much amazing stuff came out of Solaris engineering and the systems division!
> - UltraSPARC was more than a decade too late to make up for SPARC falling way behind x86_64. Sun needed to give up on SPARC, but again, vendor lock-in sounds sweet but turns out to be poison.
UltraSPARC came out in 1995, and Athlon 64 in 2003. I'm confused on how it could be late?
> - The MySQL purchase. WTF, this was horrible and stupid. The only interesting effect of this was to make Sun a target of acquisition for Oracle. [..]
I disagree with this assessment strongly. I believe Oracle wanted Java and only Java. All other Sun properties were collateral damage of the acquisition. It might be weird that Oracle now owns MySQL, but Oracle never cared about it. (Of course, both your and my assessment are mere opinions.)
Sun did really well in the academic workstation market, converting the old "VAX and a sea of terminals" into a "sea of Sun workstations".
X11, YP/NIS and NFS worked pretty well for a while, but then Microsoft came out with NT and AD, and Sun failed to respond. Like, ever. I think this was perhaps the worst software mistake Sun made.
I tried to sell XAD, the Active Directory workalike I wrote 2002-2006, to Sun. Would’ve integrated nicely with SunDS (a big part consisted of SLAPI plugins). Was in touch with Jonathan Swartz, but no dice. Sold it to Novell instead. Ironically ended up porting SLAPI to eDirectory to integrate it.
It's worse because huge chunk of Active Directory is essentially building on what MIT Athena did, which was heavily used with Sun as well - and all packaged into something that can be brought up operationally by SMB team with no previous experience
Yes, Kerberos is a direct import, although extended quite a bit. AD is a lot more than Hesiod, but AFS is a lot more than SMB. Zephyr didn't make it over.
There was an opportunity to build out graphical management tools, but NIS+ kinda got in the way, and the politics of commercial Unix in general didn't help.
The nice thing with kerberos is that it v5 has extensibility effectively built in :)
I'd say the switch to DAP/LDAP was obvious better choice than Hesiod, and GPO essentially standardizes a lot of tooling that SIPB et al built for Athena as well
Building an AD-compatible service is not that easy, and it was not easy circa 2006, because there was a lot to reverse engineer. Sure, Sun had access to its own directory service (Sun DS), MIT Kerberos, BIND, OpenLDAP, NSS, and Samba, so it had a lot of the pieces, but there was a lot of work to be done putting everything together.
The issue was not reverse engineering Active Directory to make a compatible system, it was that they didn't really capitalize in that direction. Sun has all the necessary pieces to make a compelling option at the time.
Microsoft really only wiped competition in the area when 2000 landed with Active Directory instead of the clunky previous NT Domain system.
Yes. Over-generalizing a bit -perhaps-, the issue was that any Sun division that was a profit center tended to rest on its laurels instead of innovating and disrupting. Solaris was a cost center, so Solaris engineering innovated. The DS folks were a profit center.
The disagreement had been going on for years (at least since Google launched android, perhaps before that), Google and Sun had been in “negotiations” that were publicly documented as going nowhere. All that had happened is Google implemented more “workarounds” to use less of Sun’s IP.
I actually remember media speculating that Sun didn’t want to take the next step and actually file a lawsuit, because they didn’t have enough resources to go up against Google in court.
Then Oracle bought Sun, and suddenly the unproductive negotiations transformed into a lawsuit.
IIRC the writing was on the wall by 2002. The x86 hardware was finally good enough that first there was a lurch to WinNT, then Linux once people realized NT wasn't as stable or developer friendly.
That's when I went to work for them. As far as forming my career went, it was a great move. And since it was Solaris engineering I went to, it was quite fun. But yeah, Sun was done -- it just didn't know it, and it spent almost a decade making mistake after mistake. I do think Sun could have been much more successful if senior management had made better decisions, in particular if they'd committed to x86_64 as the future.
Thanks for sharing. What do you think was the reason that the hardware part of Sun failed to keep up with competitors? Specifically why UltraSparc failed to match Pentium?
CPU design is expensive. Intel was ahead of the competition and wasn't fabbing, so Sun was a generation behind. That sort of thing.
Note that today things are quite different, and Apple can do what Sun wanted to do, but do it well. Market conditions are very different, and Apple, of course is the 800lb gorilla that Sun couldn't quite get to be. It's kinda funny: Apple and Sun weren't all that different, and Sun then and Apple now even more so. Apple just wasn't into servers, and Sun just wasn't into consumer hardware, but both wanted to vertically integrate hardware and software. Apple has a walled garden, Sun didn't. I do think these differences are relevant to why Sun failed, but the leadership differences count for a lot too: no one would compare Steve Jobs to Jonathan Schwartz, nor even to Scott McNealy -- it's just night and day. Now the differences in what Sun and Apple sought to do also evince differences in leadership.
Thanks for sharing. I know nothing about CPUs themselves so was curious. I always thought that Sun was into the more premium stuffs, but then I realized that Intel actually caught up with commodity chips. The other workstation/server companies seemed to fall into the same path unfortunately so I guess Intel/AMD was really strong back then.
I started using Sun workstations in the late 80s. I remember when their revenue reached $1b. I had a Sun on my desk as my dev machine for 7 years.
They, and the software that ran on it (Dataviews), and their customers (Westinghouse Software in my case) were all soon undone by the arrival of powerful commodity Wintel.
It's a good list, and it's supported by the anecdotal evidence from my own career: my first dev job after college was on Solaris, but that company moved away from Sun because of Sun DS.
Why do you think ZFS on MacOS would have helped in any significant way? AFAIK, and I may be wrong, but Apple doesn't have a history relying on OEMs for their software, especially something as critical as operating systems. Am I wrong? Was it different back then?
> Why do you think ZFS on MacOS would have helped in any significant way?
It would have bought Sun mind-share. In that time frame Sun was trying really hard to re-acquire mind-share it had lost to Linux due to the non-deal with Google, the brief cancellation of Solaris x86, etc. OpenSolaris was an attempt to rebuild mind-share. Was rebuilding mind-share enough? No, no way, but it was a start.
It could have led to Apple acquiring Sun, say.
> AFAIK, and I may be wrong, but Apple doesn't have a history relying on OEMs for their software, especially something as critical as operating systems. Am I wrong? Was it different back then?
Apple did the work of integrating ZFS into OS X. They believed in it. Steve Jobs wanted indemnification in light of the NetApp lawsuit. Jonathan wouldn't agree to it. Giving it could have been a betting-the-company event, but Sun needed that sort of thing.
At least they had the intelligence not to sue Google. I wonder how much money Oracle lost on that debacle. (The policy loss is big enough to them on its own! SCOTUS made the right call.)
reply