Having written a web crawler recently, it's surprising just how many misbehaving servers are actually out there.
Without proper error handling, a wide randomly seeded crawl would hit some form of malformed response or bizarre header within 15 minutes at the most. I eventually gave up on trying to parse all the myriad odd behaviours and now just dump them all onto a blacklist and move the crawler on.
This really is a cool one. The different ports makes it very suitable for handling a specific kind of problem. It might also be nice if there was a port that responded with a random selection from the errors (or with a proper response). e.g. "Port 5599: All of the above, at random"
In terms of functionality they are basically the same :) pathod has one server that behaves differently depending on the request you send. Hamms has different servers listening on different ports. I thought the interface was a little nicer that way. (I'm the author of hamms).
I tried to add some things as well that make testing easier when I am writing a client. Testing retry logic is hard for example because the client is making the same request a bunch of times in a row so you have to either mock the client request or make the server return different things based on the same request. I added a server that errors twice and then returns a 200, to make that easier to test.
Having ability to forward requests to an actual API of my service seems to be nice-to-have feature. (Especially for 5512 and 5513 ports, where actual correct 200 OK responses are expected.)
Without proper error handling, a wide randomly seeded crawl would hit some form of malformed response or bizarre header within 15 minutes at the most. I eventually gave up on trying to parse all the myriad odd behaviours and now just dump them all onto a blacklist and move the crawler on.