Currently it just fetches latest if a version is not specified. It should be pretty easy to use the current project version if available, assuming llm plugins stay in the working directory of the main call. PRs welcome :)
Once the module version is in GOMODCACHE, it's extremely fast, not even noticeable.
1Password truly doesn’t get enough credit for the choice to encrypt every vault with a high entropy secret key passed device to device. It surely costs them in UX and support load, but it would have made a breach like this essentially inconsequential.
Bitwarden truly doesn’t get enough credit for being completely open source and having independent implementations of the server code (Vaultwarden) with which the official clients are fully compatible, which I can run on a vm on a server under my desk.
In 50 years time, who knows if any of these companies will be around. But I’m pretty sure that my grandchildren, should they want to, will be able to open a gpg encrypted gzipped file (with the passphrase I’ll leave them) containing my passwords in a csv file.
I am fascinated by the idea of being 50 years from now, and doing digital archaeology more or less. So much of our actual output is now digital and stored digitally.
Given how I have experienced technology up until this point, my assumption is that everything I will create for work or for pleasure, is more or less ephemeral. It has certainly proven true for work.
I think we (or our descendants) will be surprised by the longevity of some of the file formats in use today. I would wager that it will be possible and not too unusual for regular users to open files in formats like PDF, zip or jpeg 100 years after their inception.
I've recently had to open some installer files from the mid-90s that were in a proprietary format (I forgot the name... Inno? InstallShield?) and was surprised to see that the current go-to solution is open source.
As long as an open source (or at least open specification) exist, these files will remain being openable. ...or at least until curious minds are able to crack them!
I'm not sure whether these file formats will still be in common use, but I'm fairly sure it will be trivial to find software that can read them.
Just like it's fairly easy for us to even run software from 50 years ago thanks to emulation. As long as your software run on a platform popular enough to have a good emulator. But for PDF and zip and jpeg reading software that will definitely be the case.
Yes I do think we will still have files (whatever they will be called) at some level for some purposes.
I.e, we will still be able to store and transfer sequences of bytes conforming to some specification (file format), and we will be able to attach names to those blobs in some namespace. The concept is too general to ever lose its usefulness.
There are a few key things I have learned in the third of a century that I've been working with data: Data lives longer than apps and longer than people. We will always need units of data that have their own life cycle and are reasonably self describing and self contained (i.e meaningful without resolving external references).
100 years after their inception isn’t very far from now. There are plenty of people here who will be alive in the 2080s.
Everybody has a different idea of what long term means, but I think of it as millennia from now. The kind of time frame that the Long Now Foundation talks about.
I suspect you won’t be allowed to do archeology, because the company that bought the rights to it won’t let you. If we continue the path we’re on basically nothing will be owned by us or kept safe by us. Just look at the difference from the PC era to the smartphone era. It’s all cloud based now.
I have installed the "1password-cli" package on my airgapped linux machine with no network access ('op --version' gives me 2.30.3).
If I run 'op vault list', it tells me I have to add an account. When I run 'op account add' it tries to connect to 1password's servers and won't let me proceed without internet.
I don't see how this "local client" is helping if all the auth infrastructure goes through their servers.
There might be alternatives that are better designed for that use case these days; pass and KeePassXC are popular ones, depending on the interface you want (pass is made for the cli as the primary interface).
What does "You will get your vaults locally" mean?
Is it possible to export as a file, take that with you on whatever medium (eg. USB key, CD-ROM, future isolinear chip), put it on a brand new PC you built from scratch and never connected to the internet, and open it in some kind of standalone viewer?
That’s how 1Password used to work. Not sure how much of that is still left in the system these days.
Originally it was an app with no remote component. The vault was yours to look after. Most people kept it in Dropbox to make it accessible anywhere. The vault itself actually had an html file in it that you could open in a pinch that was able to decrypt secrets (only for reading, from memory).
Actually, 1Password had local syncing where you synced the vaults between devices on a local connection (I think it was point to point WiFi, so your internet dropped off, Bluetooth was less common then). So it was bucket brigade syncing.
Dropbox came later and security minded folks were wary. Honestly, I trust 1Password sync more than an encrypted db on a general purpose cloud file sync, but maybe that’s naive.
After auth, it downloads a copy of your vaults to your device from their servers.
Super contrived, but you could probably just copy the sqlite dbs of your vault it creates locally to another PC along with the 1Password installer and it might let you sign in with just your master key.
The likelihood that someone would be able to do this in 50 years time, without your company still around? Close to zero.
Passwords, even ssh keys and passkeys, are little pieces of plain text. If you think needing a specialised sdk or cli to retrieve plain text is a good software architecture, I think we see the world quite differently.
That's the exact reason it's open source, so it would still be possible to access your data in such an event.
We clearly see things differently but I think using computers to make our lives easier is worthwhile and storing/managing our secrets securely, effectively and conveniently is better managed by software than some ad-hoc setup.
Nitpick, passkeys are not text, they are binary blobs.
“1Password anywhere” (single html file password manager) stopped working a while ago. May be 6 years back. Sure you can install a new client and use a stored folder - but compatibility lasting to your grandchildren’s time / 50 year etc - highly unlikely
Since I’m talking about reliable long term archival of critical encrypted data here, let me again ask: in your opinion, what is the likelihood that in 50 years time, with 1Password long gone, my grand children would be able to run that local 1Password client and successfully decrypt the data?
Because I feel pretty confident that gpg will still be around (though hopefully long deprecated), that gzipped files would still be able to be opened, and everyone would still be able to open a csv file. Without any specialised software, sdk or whatnot.
If this scenario doesn’t concern you, that’s fine, 20 years ago it wouldn’t have been my concern either. But the older I’ve become, the more I think about this stuff.
Close to zero. Archive is a different discipline. You need to have formats that are long lived and accessible over time. Paper is best, and it goes from there. Some electronic media archivists are fans of TIFF. It’s a field with controversy.
Pick the formats your storing and handle security at the container. This might be an encrypted system that is copied and updated over decades or a physical storage safe or box.
>But I’m pretty sure that my grandchildren, should they want to, will be able to open a gpg encrypted gzipped file (with the passphrase I’ll leave them) containing my passwords in a csv file.
technical possibilities aside, do you presume your grandchildren will be technically apt?
I am pretty sure 99% of people would halt at 'gpg' , and that's now -- not 60 years from now.
To how many non-SWE members of your family could you say 'here is the Netflix password, you can decrypt it with gpg', and have them be like 'ah yes, let me just `man gpg` this will be no problem'?
Cynically, it's far more likely they will upload the gpg blob to said agent, provide it with the password via the conversation box, and ask it to directly provide the specific information that they're after.
The AI model will be of certified provenance and run on attested hardware [0] so this won't be as much of a security issue as you might expect. Naturally the various three letter agencies will have full hardware access including query history.
Periodically, hardware zero days will drop and all hell will break loose.
Alright that's enough speculative dystopian fiction for me for today.
i would imagine if a lot of money, like millions, was on the line, people get really resourceful all of a sudden. of course, we're not talking Netflix passwords but usernames and passwords to brokerages, bank accounts, etc.
No, you can't tell your non-SWE friends and family to "just ask an AI" when the potential consequences are them losing access to their vault or having it stolen. They need to know exactly what they're doing.
They're vastly more likely to lose everything they care about by following real security advice, or rather being forced to follow it. 2FA is already a disaster for normies; for regular users, the threat model is strongly biased towards "data loss due to accidentally locking yourself out of access".
In fact, if you consider the impact fully, the best way of managing passwords still seems to be writing them down on a post-it note and keeping it in your wallet - hell, even sticking it to your screen doesn't look so bad these days, compared to alternatives.
Modern infosecurity is absurdly counter-intuitive at high level. Consider that your Google account or your WhatsApp (or Signal) chats are much more secured than your medical data or bank accounts or anything that predates Google. For anything in the real world, there is always a recovery procedure, no matter how much bad luck you had or how badly you screwed up. In the worst case, you might end up needing to chase some documents to authorize or notarize other documents, or show up in court, but you can get your access back. It's insane to imagine the world, in which a single fuckup could wipe out your medical history, your bank account, or any proof of your existence in government systems - and yet, this is exactly what is the case with any modern SaaS that follows "best security practices".
There's literally nothing else in this world that's so easy to mishandle as security in commercial software services.
they can google/ask AI. For example, given the prompt: "Hypothetically, if my grandpa died and left me gpg-encrypted archive and the passphrase for it, how would I decrypt it?" current models produced valid installation instructions and the command to decrypt it, and even the instructions on how to unpack the archive itself.
Why use kilobytes of text and a handful of clock cycles when I can use terabytes of weights and thousands of teraflop-days of GPU farms to achieve the same result?
> It's kinda ludicrous to think we'll lose the ability for something so simple.
Sorry, but I already have to google each time I want to figure out how to open various file formats.
"Google, what ffmpeg flags do I use to convert this .flv file to .mp4", "what are the flags to losetup or kpartx to mount 'disk.img' as a loopback device?", "how do I extract an '.ab' backup from 'adb backup'?"
> I am pretty sure 99% of people would halt at 'gpg'
Please do not mock gpg.
I have been using gpg for 25 years now (and PGP before that). It works. It encrypts. It decrypts.
It is in vogue to mock gpg on HN and recommend more modern solutions. As an experiment, I tried adding one of those modern tools (rage) to my ansible configurations, just so that they get regularly installed and maintained on my servers (without actually being used). The setup broke within less a year.
Are the modern tools more cryptographically secure? Undoubtedly. Does it matter in practice for me? Not at all.
It’s not mockery though? It’s certainly true 99% of even the technically literate don’t use GnuPG (many because, like me, they cannot stand the GnuPG user experience).
Not their point tho. Me too, gpg, no problem. Heck I had the "'smuggled' out of the country as a book" pgp back when.
My kids? I really am not so sure at all. Still too early to tell for sure but so far I don't think any will be as technically savvy as I am. I really doubt they'd know what GPG (or PGP) are and how to use it.
Presumably if you're leaving your kids an encrypted file with valuable stuff in it they won't be so illiterate that they can't figure out how to open it. Most people don't know gpg because they have no motivation to learn it not because they're incapable of learning it.
No motivation, sure. But do they even care? Most people just don't care about this stuff. You try to get them to care and they "say, yeah, okay, yeah, I know" as they roll their eyes and go back to scrolling Instagram or whatever.
The apathy is real and I don't see it getting better.
Curious, how is Excel encryption? That may be a more approachable format than CSV GPG, and though technically the CSV GPG is more simpler, it may be less familiar to users in 100 years. Excel will still be around ;)
It's such a pity that bitwarden's client doesn't work offline for modifying vaults (need to be online to be able to access the server implementation). I would switch from my old local vault 1password in an instant.
I just have KeePass in a syncthing folder with a trigger to sync on open.
Technically I think I could drop the trigger if the desktop app would open by making a temporary file copy and syncing back (ironically Keepass2Android is very good at this).
I still rely on a gpg encrypted text file for storing my passwords, too. 25 years of that and it's second nature. No other solution has ever appealed to me.
I’m autofilling usernames and passwords from 1Password’s browser extension probably 100+ times per workday. Are you manually copying and pasting anywhere near that amount? I think I would be miserable with that setup for anything beyond very light use.
That, and it's better in most functional and polish regards than LastPass. I haven't used 1Password, so I can't compare those two directly, but I'd strongly recommend BitWarden over LastPass as far as those two are considered.
I have my own beef with 1Password, but having used both Bitwarden and 1Password, I still find 1Password to be the better UX and more secure solution. Bitwarden is also worse at filling with their browser extension, rather significantly. That said, 1Password's Safari support with multiple profiles is... frustrating... at best.
I’d strongly prefer an open source and selfhostable option, but each time I’ve evaluated Bitwarden in the past, it was a big enough downgrade from 1Password that I didn’t think switching was a good option.
If the experience ever becomes as seamless, I’ll be switching.
That is particularly true for anything dealing with security. I evaluated both BitWarden and 1Password when we wanted to migrate away from LastPass. My recommendation was to eventually go with BW. Its open-source nature was a factor, but for a corporate use the UX factors were even more prominent.
Over a course of a month, I ran into several subtle footguns with 1P. Search included only some of the fields. Password reset/rotation flow was easy to mess up (thanks to the confusing + inconsistent "copy field" functionality) and get into a situation where the generated password that was stored in the vault was different from the one that was set: in my tests there was 50/50 chance of accidentally regenerating the password before the vault storage step after submitting the new one for a remote service.
There were a whole load of "features" that didn't make any sense. The UI for 1P was a real mess. The feeling I got from it was that their product had been captured by Product Managers[tm] desperate to justify their own existence by shipping ever more Features[tm] without considering the impact on the core functionality.
BW's UI is by no means perfect, and their entry editing flow is far from ideal. But at least most of the actual usability snags in their browser extension have a common workaround: pop the BW overlay out from the browser, into a separate window. Their open-source nature and availability of independent implementations mean that there will be alternatives, should BW go down the same features-features-and-more-antifeatures hellhole in their race to eventually appease their VC backers.
Sounds like our experience with it could not be more different.
> The UI for 1P was a real mess.
In what way? You described how you feel about the UI, but I’m curious about actual specifics.
It’s entirely possible that I’m just too accustomed to it because I’ve been using it for many years, but what you’re describing is how I felt about Bitwarden.
I can completely see choosing BW in a corporate setting for a host of other reasons. But for me personally, the priority is a tool that gets out of my way and just works.
The tool that has done that is 1P.
> Less is more.
That really depends. If less means that the password manager doesn’t get used, then less is less.
I check BW every so often but it always feels less polished UI wise. For all the complaints people had about 1P moving to electron, it’s UX is still the best out there.
I'm confused why some companies (including Amazon and Steam) insist on family features. The mental model behind this is more prescriptive than descriptive - it doesn't match to how users and their families function; rather, it insists on some activities to a) exist in family, and b) be not allowed outside of family.
Or simply: how many people have actual family listed in their Steam / Amazon "family sharing"?
What do you mean about prescribing and insisting? I’m not sure I understand your questions about family sharing and the mental model.
I use family sharing with actual family for my Steam account and all video streaming services. Am I weird? The reason is because streaming services allow sharing under a single paid account, and my wife & kids don’t want to pay for separate accounts, and don’t want to have to authenticate separately on shared devices (TVs, game consoles, iPads, etc). Steam family sharing works across different Steam accounts, and sharing a single account doesn’t work, so Steam isn’t particularly relevant to the discussion of family sharing of passwords. Steaming accounts, on the other hand, all assume they’re being used by a whole family, and the main reason is because of shared devices; the family TV itself logged in. So, they all offer profiles under a single account. Netflix clarifies that family sharing means the people in a single household, maybe others are similar.
We use password family sharing as well. My wife and I share bank and credit card accounts. My wife needs my accounts sometimes to do certain things — you might be surprised how many banks do not offer joint accounts and still treat wives as second class citizens. We share the Netflix & Amazon accounts with the kids so they can use them. I pay for a 1Password family account and share it with my aging father who’s been losing passwords. These things are all pretty useful for me.
I guess you’re making me wonder why someone wouldn’t make a family sharing feature, when it solves real problems and users are asking for it?
I don’t have amazon or steam so don’t know how any of that works. But for a password manager, family sharing is extremely useful.
Bitwarden doesn’t have families per se, it’s got “organisations”. You can setup unlimited number of organisations and users can get invited and join them. Which is very handy for example my wife and I can login and order our groceries from the supermarket using the same account. Or that we can both login and use our electricity company’s web portal which only allows one account per household. All without needing to send each other passwords and updated passwords back and forth.
I have nothing against sharing per se. My issue is with the family nomenclature. In your case it might align perfectly, but for myself and most people I know, it's not the case. That is, the set of people to share a Netflix subscription with, share Steam library with, share Kindle library with, share passwords to various web services, including utility companies, are only partially overlapping, and do not align perfectly with the idea of "family" or "household".
This seems pedantic. I am trying to wrap my head around why "family sharing" is an issue here. You want to share with someone, use family sharing I don't see what the issue is.
Their comment made me laugh, agreed, open source is really is that big of a perk. IMO especially for something security-related (though 0day is always possible)
I think the assumption something is open source being a perk is naïve - in the end I still have to trust so many elements that actually bring me the platform that BW’s openness doesnt matter.
Proton Pass truly doesn’t get enough credit for being completely open source, more user friendly, and hosted outside the US (wouldn’t want to lose access to your vault [1]).
Proton pass is a poor man’s attempt at a password manager, with horrible user experience (oh we thought just a browser extension was enough!) and random limitations to fit in with Proton’s tortured business model.
I was a Protonmail founding member. I used and evangelised them for years until I realised that they are more interested in chasing the next shiny thing (hey we have a crypto wallet now!) instead of fixing longstanding bugs and performance issues in their mail client.
As for hosted outside the US, I’m pretty sure the vaultvarden instance running under my desk is also hosted outside the US (unless I’ve somehow been magically transported to the US). Plus, I get to physically lock the door when I leave the house and my cat usually sleeps on top of the sever which adds a level of furry protection which proton pass could never achieve
Oh, impressive how browser extensions can live outside of browsers now. And on mobile too!
After having used Bitwarden for more than 4 years, I only switched last week, so I'm still in the honeymoon phase. But it has everything I used in Bitwarden and more, most notably all the usability features that I was missing in Bitwarden.
Don't think you can self-host it, but that would also pretty much defeat the user-friendly aspect. You don't need the server source if the vault is client-side encrypted, besides you would get zero guarantees that what they show and what they're actually running are the same thing.
Wouldn't it also make you lose everything in a recovery scenario? If all your computers are lost in a fire or flood, you would lose the recovery key, and having your password would not be enough to recover your database. I use keepassxc with a somewhat long password with a high PBKDF iterations count, which would not require having any devices in the event of a loss.
There is an option to print out recovery info. A sheet with a QR code and a space for you to write your password (or not, if you don't trust keeping those 2 things in one place). That paper can go in a safe deposit box, with a trusted family member / friend, or in some cloud service you'd still have access to. The QR code + your password allow for recovery.
Depending on your usage, loosing your password can be irreversible. That'd lock you out of your encrypted email and storage and will take you months to recover your account on some platforms.
I'm not involved in the OpenSSL/CNG-based Microsoft Go fork.
I've managed and implemented—along with Daniel McCarney, Roland Shoemaker, and Russ Cox—the native upstream Go validation mentioned in the intro, which is shipping in Go 1.24 and will be certified on Linux (amd64, arm64, ppc64le, s390x), Windows (amd64, arm64), macOS (arm64), and FreeBSD (amd64). The Linux operating environments were funded by various stakeholders, the rest were funded by Geomys for the benefit of the Go community.
There are some details now at https://go.dev/doc/security/fips140, but we're going to write a proper blog post once the module gets on the CMVP In Process list.
tl;dr is that it should soon take a single environment variable to transparently build against a FIPS 140-3 validated module which is just a slightly out of date version of the same Go standard library everyone else is using. (AFAIK this is the first non-JVM memory safe FIPS 140 module!)
Is there going to be a way to turn FIPS on as a requirement without having compiled the binary against the FIPS-140 module?
Even now it has been a lot of trouble trying to get vendors to provide us a way to rebuild/repackage their Golang binaries using the FIPS-140 support from RHEL, and if it is still the case that they need to build one-offs, teams will still need to rebuild/repackage a lot of Golang tools just to be FIPS compliant because it's not as simple as setting a flag like it is for OpenSSL.
> When GODEBUG=fips140=only is used, in addition to the above, cryptographic algorithms that are not FIPS 140-3 compliant will return an error or panic. Note that this mode is a best effort and can’t guarantee compliance with all FIPS 140-3 requirements.
Will need to be more complete/certain in order to pass certification? My gut says it would based on the requirements, but I don't know how the auditors would respond in practice.
I just wanted to say that we've adopted many of your projects at $DAYJOB, and it has been one of the best technical decisions we've made. So far I've used the Go crypto package, age, and more recently xwing. It all works flawlessly, and at every turn we have happy surprises (smooth cross-compilation, the Age plugin framework, typage, fast xwing revisions, your blog posts).
I'm so glad this is making its way in to Go properly now, and sort of baffled that this wasn't treated as a priority sooner by Google, given that they're operating within FIPS environments.
There's extremely small (like 2⁻²⁰, the chance of winning $50 000 with a $2 Powerball ticket [1]), and then there's cryptographically negligible (like 2⁻¹²⁰, the false negative chance of our primality test). The chance of something cryptographically negligible happening is about the same as the chance of the attacker guessing your key on the first try, or of a cosmic ray flipping a CPU flag inverting the result of "if !signature.verified { return err }".
I happened to look at this recently, and while I understand the argument (but not the math) of having to do fewer Miller-Rabain rounds, why would you do so in PRACTICAL settings? Unlike ECC you're likely only generating long term keys, so shorter key generation time seems like a bad tradeoff. Composite candidates are going to be rejected early, so you're (with high probability) not doing expensive calculations for most candidates. My reading of [BSI B.5.2](https://www.bsi.bund.de/SharedDocs/Downloads/EN/BSI/Publicat...) confirms this.
Of course random bit flips could interfere, but other measures should thwart this in high-stakes environments (at least to some degree).
The number of Miller-Rabin rounds has to be bounded, so if you're not going to base your bound on reaching a cryptographically negligible change of false positives, what are you going to base it on? Should we do 10? 15?
The problem with "x should be enough, but why not do more?" arguments is that they can be applied recursively, and never answer the question "ok so when should we stop?"
The BSI recommendation I linked says 60 times here (considering the worst case and the security level they're targeting). Just wondering why we'd want to do less for a (presumably rare) one-time operation.
The performance of this can matter in some scenarios. In embedded systems, smart cards etc., generating the primes can take a significant amount of time (15-20 seconds typical) even with the 'low' number of iterations. Higher iterations means the user will have to wait even longer. Actually, in such systems it is not unusual to see occasional time-out errors and such when the smart card is unlucky with finding the primes. The timeout value may be a fixed, general value in a part of the stack difficult to change for the specific call.
Another case is short-term keys/certificates, where a fresh key pair is generated, and cert issues, for each and every signature. This setup makes revocation easier to handle (the certificate typically has a lifetime of a few minutes so it is 'revoked' almost immediately).
There are also scenarios where the keys are generated on a central system (HSM's etc.) to be injected into a production line or similar. There performance can matter as well. I worked on a system where the HSM's were used for this. They could typically generate an RSA key in 1-2 seconds, so 12 HSM's were needed to keep up with demand - and this was again with the reduced number of rounds.
Thanks for the reply (and it definitely answers my original question). For both short-lived keys, and where production time matters it makes perfect sense. I was mostly thinking about the scenario where you're generating a key for e.g. a long-lived certificate (maybe even a CA) or high-stakes PGP stuff. Just seems like you'd want spend a few more seconds in that case.
Why not do 120 then? We can show that the chance of false negative of 5 rounds is cryptographically negligible, so 5, 60, and 120 are all the same. If the only argument for 60 is that it's more and this is a really important rare operation, doesn't it also apply to 120?
I'm not trying to be glib, there is no rational stopping point if we reject the objective threshold.
I don't pretend to understand all the involved math, but what I'm trying to say is that as far as I understand T rounds gives 4^-T probability that we've chosen a "bad" prime (really composite) per normal Miller-Rabin in the worst case. Doing ~5 rounds has been shown to be enough to choose a good prime candidate, under most (very probable!) conditions when we get it a random, and thus the argument is that that ~5 rounds is fine. We agree so far?
I'm just asking, why not run the conservative 60 round test, rather than ~5 when you're doing a very rare, one time, key generation? I understand that it's very unlikely to reject any numbers, but at least BSI thinks it's worth it for important keys.
If I understand the recommendation right, you wouldn't do 60 for a 2048 bit key and then 120 for 4096, rather 61 rounds would be enough for 4096 if 120 is alright for 2048.
You're asking why not apply the formula for adversarially selected candidates even if we are randomly selecting candidates. There is simply no reason to, except "maybe we made a mistake" but then why would we not think we made a mistake also in calculating the 1/4 value, or in any other part of the code?
Phrased another way, do you have an argument for why run the conservative 60 round test, instead of asking for an argument for why not run it?
Again, you are "very unlikely" to win the Powerball jackpot. Rounds 6-60 have a cryptographically negligible chance of rejecting a composite. It's different, otherwise we'd have to worry about the "very unlikely" chance of the attacker guessing an AES-128 key on the first try.
(I don't follow you on the key sizes, if you apply the 1/4 probability, the candidate size is irrelevant.)
Thanks, understand what you mean. I probably botched the 1/4 probability thing, was thinking 4^-60 gives 2^-120 bit assurance (roughly, security margin in my mind), and an extra round would quadruple it, but doesn't work that way I realize.
As I understand it, the number of rounds needed (for a given acceptable failure probability) goes down the larger the number is. Note (very importantly) that this is assuming you are testing a RANDOM integer for primality. If you are given an integer from a potential malicious source you need to do the full number of rounds for the given level.
> Rule of thumb: Want a 1024-bit prime? Try 1024 1024-bit candidates and you'll probably find one.
Where probably is 76% [1], which is not that high depending on what you are doing. For example, you wouldn't be ok with GenerateKey failing 24% of the time.
To get a better than even chance, 491 [2] 1024-bit candidates are enough.
Hah, I wrote the crypto/rsa comments. We take Hyrum's Law (and backwards compatibility [1]) extremely seriously in Go. Here are a couple more examples:
- We randomly read an extra byte from random streams in various GenerateKey functions (which are not marked like the ones in OP) with MaybeReadByte [2] to avoid having our algorithm locked in
- Just yesterday someone reported that a private ECDSA key with a nil public key used to work, and now it doesn't, so we probably have to make it work again [3]
- Iterating over a map uses a randomized order to avoid exposing the internals
- The output of rand.Rand is considered part of the compatibility promise, so we had to go to great lengths to improve it [4]
- We discuss all the time what commitments to make in docs and what behaviors to disclaim, knowing we can never change something documented and probably something that's not explicitly documented as "this may change" [6]
The map iteration order change helps to avoid breaking changes in future, by preventing reliance on any specific ordering, but when the change was made it was breaking for anything that was relying on the previous ordering behaviour.
IMO this is a worthwhile tradeoff. I use Go a lot and love the strong backwards compatibility, but I would happily accept a (slightly) higher rate of breaking changes if it meant greater freedom for the Go devs to improve performance, add features etc.
Based on the kind of hell users of other ecosystems seem willing to tolerate (cough Python cough), I believe I am not alone in this viewpoint.
Data point of one, but I've been using Go since 2012 and would drop it instantly if any of the backwards compatibility guarantees were relaxed.
Having bugs imposed on you from outside your project is a waste of time to deal with and there are dozens of other languages you can pick from if you enjoy that time sink. Most of them give you greater capabilities as the balance.
Go's stability is a core feature and compensates for the lack of other niceties. Adding features isn't a good reason to break things. I can go use something else if I want to make that trade.
Respectfully, I don’t think you would just pack up and leave. The cost of switching to an entirely different language—which might have even worse backwards compatibility issues—is significantly higher than fixing bugs you inadvertently introduced due to prior invalid assumptions.
That's a bit bold when you know nothing about me, but sure.
I exist in a polyglot environment and we use Go for things that we expect to sit and do their job for years without modification.
Nothing more annoying with these than needing to update a runtime to patch a CVE and suddenly needing to invest two weeks to handle all the breaking changes. Go lets us take 5 minutes to bump the version number in the Dockerfile and CI configs and move on to more important work.
I'm not suggesting we'd go rewrite all of those if Go relaxed its guarantees but we'd stop picking it to write new things in and it would slowly disappear as we decommission the existing services over the years.
Every language and its environment has issues. Switching always introduces a new set of problems, some of which could be worse, and many of which you won't have anticipated when you encounter them.
Breaking API changes in a minor version update sucks and is often an unexpected time sink, and often mandatory because it has some security patch, critical bug fix, or something.
Breaking API changes in a major version update is expected, can be planned for, and often can be delayed if one chooses.
The map iteration order was always "random", but imperfectly so prior to Go 1.3. From memory, it picked a random initial bucket, but then always iterated over that bucket in order, so small maps (e.g. only a handful of elements) actually got deterministic iteration order. We fixed that in Go 1.3, but it broke a huge number of tests across Google that had inadvertently depended on that quirk; I spent quite a few weeks fixing tests before we could roll out Go 1.3 inside Google. I imagine there was quite a few broken tests on the outside too, but the benefit was deemed big enough to tolerate that.
Breaking iteration order was also well established as a valid move. Several other languages had already made a similar change, much later in their own lifecycle than Go did. That helps a lot, because it shows it is largely just an annoyance, mostly affecting tests.
I'd consider stuff like that part of the opinion the language has. Go's opinion is that backwards compatibility at all reasonable cost is a priority.
When it comes to ecosystems, the opinions have trade-offs. I would say that Go's approach to dependencies, modules and workspaces is one of those. As a language it mostly stays out of your way, but correcting imports because it pulled in the wrong version, or dealing with go.mod, go.work and replace directives in a monorepo, gets old pretty fast (to the extent it's easier to just have a monorepo-wide go.mod with literally every dependency in it). At least it's an improvement over having to use a fairly specific directory structure though.
Java 5 was a fun upgrade for a lot of people because it caused JUnit tests to run in a different order. Due to hashtable changes altering the iteration of the reflected function names.
> We randomly read an extra byte from random streams in various GenerateKey functions (which are not marked like the ones in OP) with MaybeReadByte [2] to avoid having our algorithm locked in
You don't seem to do that in ed25519. Back before ed25519.NewKeyFromSeed() existed, that was the only way to derive a public Ed25519 key from a private key, and I'm pretty sure I've written code that relied on that (it's easy to remember, since I wasn't very happy about it, but this was all I could do). The documentation of ed25519.GenerateKey mentions that the output is deterministic, so kudos for that. It seems you've really done a great job with investigating and maintaining ossified behavior in the Go cryptography APIs and preventing new ones from happening.
The nil key case really makes me wonder how sane it is to support these cases. You will be forced to lug this broken behavior with you forever, like the infamous A20 line (https://en.wikipedia.org/wiki/A20_line).
If a private key is available, the public key can be derived from the private key using scalar multiplication. This is how ecdsa.GenerateKey works by itself - it first generates a private key from the provided random byte stream and then derives a public key from that private key.
I don't see how this can be a security risk, but allowing a public key that has a curve but a nil value is definitely a messy API.
As a user of your code this is true, and I'm very grateful indeed that you take this approach.
I would add as a slight caveat that to benefit from this policy, users absolutely must read the release notes on major go versions before upgrading. We recently didn't, and we were burnt somewhat by the change to disallow negative serial numbers in the x509 parser without enabling the new feature flag. Completely our fault and not yours, but I add the caveat nevertheless.
We have gotten a liiiiittle more liberal ever since we introduced the new GODEBUG feature flag mechanism.
I've been meaning to write a "how to safely update Go" post for a while, because the GODEBUG mechanism is very powerful but not well-known and we could build a bit of tooling around it.
In short, you can upgrade your toolchain without changing the go.mod version, and these things will keep working like they did, and set a metric every time the behavior would have changed, but didn't. (Here's where we could build a bit of tooling to check that metric in prod/tests/CLIs more easily.) Then you can update the go.mod version, which updates the default set of GODEBUGs, and if anything breaks, try reverting GODEBUGs one by one.
Breaking changes in major version updates is a completely normal thing in most software and we usually check for it. Ironically the only reason we weren't previously bothering in go is that the maintainers were historically so hyper-focused on absolute backwards compatibility that there were never any breaking changes!
> Does it know which part of a dependency has a vulnerability and check, if the execution reaches _that_ part?
Yes, govulncheck does symbol-level reachability static analysis, and the vulndb is manually annotated with affected symbols for each vulnerability.
(So glad to see a comment about this at the top, I have sometimes feared we made a mistake in designing a low-noise vulnerability scanner, because I've often seen complaints that "it doesn't work" because it doesn't show as many vulnerabilities as its more popular, less accurate alternatives.)
Exactly. Please DO NOT mess with protocols, especially legacy critical protocols based on in-band signaling.
HTTP/1.1 was regrettably but irreversibly designed with security-critical parser alignment requirements. If two implementations disagree on whether `A:B\nC:D` contains a value for C, you can build a request smuggling gadget, leading to significant attacks. We live in a post-Postel world, only ever generate and accept CRLF in protocols that specify it, however legacy and nonsensical it might be.
(I am a massive, massive SQLite fan, but this is giving me pause about using other software by the same author, at least when networks are involved.)
HTTP is saved here because headers aren't allowed to contain control characters. A server that is strict enough to only recognize CRLF will hopefully also be strict enough to reject requests that contain invalid characters.
Hopefully is not a good word to see in a argument that a software proposal is secure.
Myself, I've written an HTTP server that is strict enough to only recognize CRLF, because recognizing bare CR or LF would require more code†, but it doesn't reject requests that contain invalid characters. It wouldn't open a request-header-smuggling hole in my case because it doesn't have any proxy functionality.
One server is a small sample size, and I don't remember what the other HTTP servers I've written do in this case.
This would be more persuasive if HTTP servers didn't already widely accept bare 0ah line termination. What's the first major public web site you can find that doesn't?
Going down a list of top websites, these URLs respond with HTTP 200 (possibly after redirections) when sent an ordinary HTTP/1.1 GET request with 0D0A line endings, but respond with HTTP 400 when sent the exact same request with 0A line endings:
Most of these seem pretty major to me. There are other sites that are public but responded with an HTTP 403, probably because they didn't like the VPN or HTTP client I used for this test. (Also, www.apple.com is tolerant of 0A line endings, even though its other subdomains aren't, which is weird.)
My apologies, I was using a client which kept the connection alive between the 0D0A and 0A requests, which has an effect on www.pinterest.com. Rerunning the test with separate connections for 0D0A and 0A requests, www.pinterest.com and phys.org are no longer affected (I've removed the two from the list), but all other URLs are still affected.
Make sure you're contacting hhs.gov and not www.hhs.gov, the www. subdomain reacts differently.
$ printf 'GET / HTTP/1.1\r\nHost: hhs.gov\r\n\r\n' | nc hhs.gov 80
HTTP/1.1 302 Found
Date: Mon, 14 Oct 2024 01:38:29 GMT
Server: Apache
Location: http://www.hhs.gov/web/508//
Content-Length: 212
Content-Type: text/html; charset=iso-8859-1
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="http://www.hhs.gov/web/508//">here</a>.</p>
</body></html>
^C
$ printf 'GET / HTTP/1.1\nHost: hhs.gov\n\n' | nc hhs.gov 80
HTTP/1.1 400 Bad Request
Date: Mon, 14 Oct 2024 01:38:40 GMT
Server: Apache
Content-Length: 226
Connection: close
Content-Type: text/html; charset=iso-8859-1
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
</body></html>
This exercise was about demonstrating that our security can't rely on making sure there's a carriage return in HTTP line termination, because there is no such norm. See the root of the thread, where I asked the question.
Oh, I agree it's about that too, but my point is you've already volunteered more time and resources investigating the situation than most companies would be willing to spend.
As the parent mentioned, it's security critical that every HTTP parser in the world - including every middleware, proxy, firewall, WAF - parses the headers in the same way. If you write a HTTP parser for a server application it's imperative you don't introduce random inconsistences with the standard (I can't believe I have to write this).
On the other hand, as a client, it's OK to send malformed requests, as long as you're prepared that they may fail. But it's a weird flex, legacy protocols have many warts, why die on this particular hill.
Wouldn't the safest thing, security-wise, to fail fast on bare 0ah?
As a web server, you may not know which intermediate proxies did the request traverse before arriving to your port. Given that request smuggling is a thing, failing fast with no further parsing on any protocol deviations seems to be the most secure thing.
If you expect to be behind a reverse proxy that manages internal headers for you (removes them on incoming requests, and adds them based on internal criteria) then accepting bare 0x0a newlines could be a security vulnerability, as a malicious request could sneak an internal header that would not be stripped by the reverse proxy.
That was already motivated by Postel's Law. It's a step beyond to change what the strict form is; relying on the same to justify unilaterally transposing the form is asking too much of middlebox implementations of just about any line-oriented protocol, and possible violates Postel's Law itself by asserting the inverse.
I don't believe in Postel's Law, but I also don't believe in reverential adherence to standards documents. Make good engineering decisions on their own merits. This article is right: CRLF is dumb. You know who agrees with me about that? The IETF, in their (very old) informational RFC about the origins of CRLF in their protocols.
Yes, CRLF is dumb. Trying to justify the problem seems unnecessary, it's widely acknowledged. A productive inquiry looks at why fixing it didn't happen yet. Don't confuse that line of thought for calling for more failure.
This is unrealistic, though:
> I don't believe in Postel's Law
All the systems around us that work properly do believe in it, and they will continue to do so. No-one who writes MTAs or reverse proxies &c is gonna listen to the wolves howling at the moon for change when there's no better plan that "ram it through unilaterally". Irrespective of what any individual may believe, Postel's Law remains axiomatic in protocol design & implementation.
More constructively, it may be that line-oriented protocols will only move towards change when they can explicitly negotiate line termination preferences during the opening handshake/banner/key exchange etc, which inevitably means a protocol revision in every case and very careful consideration of when CRLF is passed through anyway (e.g. email body).
> As the parent mentioned, it's security critical that every HTTP parser in the world - including every middleware, proxy, firewall, WAF - parses the headers in the same way. If you write a HTTP parser for a server application it's imperative you don't introduce random inconsistences with the standard (I can't believe I have to write this).
No it isn't, at least not critical to all those parsers. My HTTP server couln't care less if some middle boxes that people go through are less or more strict in their HTTP parsing. This only becomes a concern when you operate something like a reverse proxy AND implement security-relevant policies in that proxy.
Hrm, this is what I get for logging in to HN from my phone. It’s possible I am confusing this with one of the other exploitable HTTP/1.1 header parser alignment issues.
Maybe this was so widespread that ~everything already handles it because non-malicious stuff breaks if you don’t. In that case, my bad, but I still would like to make a general plea as an implementer for sticking strictly to specified behavior in this sort of protocols.
We're talking about servers and clients here. The best way to ensure things work is to adhere to an established protocol. Aside from saving a few bytes, there doesn't seem to be any good reason to deviate.
I'm saying the consistency that Filippo says our security depends on doesn't really seem to exist in the world, which hurts the persuasiveness of that particular argument in favor of consistency.
But no one expects 0ah to be sufficient. Change that expectation, and now you have to wonder if your middleware and your backend agree on whether the middleware filtered out internal-only headers.
Yeah, I'm not certain that this is a real issue. It might be? Certainly, I'm read in to things like TECL desync. I get the concern, that any disagreement in parsing policies is problematic for HTTP because of middleboxes. But I think the ship may have sailed on 0ah, and that it may be the case that you simply have to build HTTP systems to be bare-0ah-tolerant if you want your system to be resilient.
But what's bare-0ah-tolerant? Accepting _or_ ignoring bare 0ah's means you need to ensure all your moving parts agree, or you end up in the "one bit thinks this is two headers, others think it's one header".
The only situation where you don't need to know two policies match is when one of the policies rejects one of the combinations outright. Probably. Maybe.
EDIT: maybe it's better phrased as "all parts need to be bare-0ah-strict". But then it's fine if it's bare-0ah-reject; they just need to all be strict, one way or the other.
The exploit is that your request went through a proxy which followed the standard (but failed to reject the bare NL) and the client sent a header after a bare NL which you think came from the proxy but actually came from the client - such as the client's IP address in a fake X-Forwarded-For, which the proxy would have removed if it had parsed it as a header.
This attack is even worse when applied to SMTP because the attacker can forge emails that pass SPF checking, by inserting the end of one message and start of another. This can also be done in HTTP if your reverse proxy uses a single multiplexed connection to your origin server, and the attacker can make their response go to the next user and desync all responses after that.
There is very good reasons not to deviate as mismatch in various other things that can or are not on the path can affect things. Like reverse proxies, load balancers and so on.
Gunicorn expects `\r\n` for lines (see gunicorn/http/message.py:read_line), though it's possible that every middleware that is in front of gunicorn in practice normalizes lines to avoid this issue.
What a weird reaction. Microsoft’s use of CRLF is an archaic pain in the ass. Taking a position that it should be deprecated isn’t radical or irresponsible — Microsoft makes gratuitous changes to things all of the time, why not this one?
Hipp is probably one of the better engineering leaders out there. His point of view carries weight because of who he is, but should be evaluated on its merits. If Microsoft got rid of this crap 30 years ago, when it was equally obsolete, we wouldn’t be having this conversation; if nobody does, our grandchildren will.
No one is talking about Microsoft and whatever it does on its platform, the parent comment is about network protocols (HTTP, SMTP and so on..).
I understand that it is tempting to blame Microsoft for \r\n proliferation, but it does not seem to be the case - the \r\n is comes from the era of teletypes and physical VT terminals. You can still see the original "NL" in action (move down only, do not go back to start of line) on any Unix system by typing "(stty raw; ls)" in a throw-away terminal.
The author of the post specifically addressed this:
“Today, CR is represented by U+000d and both LF and NL are represented by U+000a. Almost all modern machines use U+000a to mean NL exclusively. That meaning is embedded in most programming languages as the backslash escape \n. Nevertheless, a minority of machines still insist on sending a CR together with their NLs”
Who is the “minority”?
He also takes the position that the legacy behavior is fine for a tty, as it’s emulating a legacy terminal.
CRLF was the correct way to implement a new line the way we think of it now, because teletypes and typewriters considered the “return to the 0th column” and “go to the next line” to be different things that are each valid on their own.
CRLF was the standardized way to implement “go down one line and return to column zero” and they’re the only ones who implemented new lines correctly at the outset.
Blaming Microsoft now, because they like backwards compatibility above almost everything else, is misplaced and myopic.
Additionally it is also dishonest to bring Microsoft into the discussion like that. The discussion revolved around _standardized_ network protocols, which is entirely unrelated to MS-DOS text formats.
He's not arguing for deprecating it. He's arguing for just not complying and hoping for the best. He explicitly says so right in the article.
That is never the right approach. You intentionally introduce a problem you expect others to fix. All because he doesn't like 0x0d. The protocol is what it is. If you want to make more sane decisions when designing a new protocol (or an explicitly newer version of some existing one) then by all means, go for it. But intentionally breaking existing ones is not the way to go.
Took me a second to get what was going on here, but basically the idea is that you middleware might not see `C:D`, but then your application _does_ see `C:D`.
And given your application might assume your middleware does some form of access control (for example, `X-ActualUserForReal` being treated as an internal-only header), you could get around some access control stuff.
Not a bytes-alignment thing but a "header values disagreement" thing.
This is an issue if one part of your stack parses headers differently than another in general though, not limited to newlines.
He does not want your code anyway, sqlite is public domain. this has several implications. One of which is the author wants nothing from you. Note that public domain is fundamentally different than the usual method of releasing code, which is to issue a license to distribute a copyright protected work. Putting a thing into the public domain is to renounce any ownership over the thing.
I think that the proper spirit of the thing is that if you have patches to sqlite is to just maintain them yourself. if you are especially benevolent you will put the patches in the public domain as well. and if they are any good perhaps the original author will want them.
In fact the public domain is so weird, some countries have no legal understanding of it. originally the concept was just the stance of the US federal government that because the works of the government were for the people, these works were not protected by copyright, and could be thought of as collectively owned by the people, or in the public domain. Some countries don't recognize this. everything has to be owned by someone. and sqlite was legally unable to be distributed in these countries, it would default to copyright with no license.
I still have high-value passwords and CLI credentials in passage + age-plugin-yubikey.