A way to deal with the problem is to make a reference implementation instead of a specification document. A document can also be written for convenience, but if code and documentation disagree, short of an obvious bug, the code is right and the docs need to change to reflect that.
And it is actually a common practice.
This way, nothing is vague or missing. If you wonder what is going to happen if a certain field has a certain value, run the code and see what happen, it may not be great, but you know exactly what to expect. With code, you also have access to all the developer tools: compilers, linters, test frameworks, debuggers, coverage analyzers, fuzzers,... and even provers. Ideally the reference implementation should be open source with a permissive license, be written with readability in mind, and come with a good test suite.
The problem with zip as described in the article is that the specs are half-assed and the real test for a correct zip file is that it should work with pkzip. But we can't really use pkzip as a reference either because it is proprietary.
I think this is how the problem was created in the first place: the authors wrote the reference implementation, but then didn't test it enough, didn't think of all possible ways things can go wrong.
Also, I'm not saying that this necessarily results in bad formats. Eg. I don't particularly like Thrift, but it's kind of OK in the sense that it doesn't do much weird stuff unexpectedly. I don't believe they had any formal way to verify that their plan is going to work, they were "lucky" because they had to work with Protobuf a lot and figured some of its shortfalls before they wrote their own.
So, unless you have some sort of a mechanical way to verify that your goals are achieved by the format you are creating, it's going to work just in the same way how any given C program won't corrupt memory: you might get lucky, or after a lot of testing you'll conclude that it's very unlikely that the program will corrupt memory, but you can never be sure.
And, really, I don't blame the authors. I'm not saying they were lazy, or didn't pay enough attention. It's just hard to do when you don't have a watchdog who will absolutely not allow any and all transgressions. I wouldn't count on myself to do that w/o such a watchdog -- I don't have that kind of confidence even after implementing many different formats used for similar purpose.
And it is actually a common practice.
This way, nothing is vague or missing. If you wonder what is going to happen if a certain field has a certain value, run the code and see what happen, it may not be great, but you know exactly what to expect. With code, you also have access to all the developer tools: compilers, linters, test frameworks, debuggers, coverage analyzers, fuzzers,... and even provers. Ideally the reference implementation should be open source with a permissive license, be written with readability in mind, and come with a good test suite.
The problem with zip as described in the article is that the specs are half-assed and the real test for a correct zip file is that it should work with pkzip. But we can't really use pkzip as a reference either because it is proprietary.