Hacker News new | past | comments | ask | show | jobs | submit login

This is not against the principle of fuzz testing. This is to say that the author doesn't really know the reality of testing and is very quick to point fingers. It's easy to tell in retrospect that this particular aspect should've been tested. It's basically impossible to find such defects proactively.



I've read both messages and I'm still unsure on how fuzzy testing may have not brought up similar edge cases.

We literally talking about a parser shutting down an entire system rather than reporting malformed data.

Considering this is a "one in 15M cases" it seems to me that fuzzy testing would've caught this and probably more bugs in a short time span.


Easy for me to say in retrospect, but IMO this is a textbook example of where you should reach for fuzz testing; it’s basically protocol parsing, you have a well-known text format upstream and you need to ensure your system can parse all well-formed protocol messages and at very least not crash if a given message is invalid in your own system.

Similarly with a message queue, handling dead letters is textbook stuff, and you must have system tests to verify that poison pills do not break your queue.

I did not think the author was setting unreasonable expectations for the a priori testing regime. These are common best practices.




Consider applying for YC's Summer 2025 batch! Applications are open till May 13

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: