I dunno. I think a program that produces subtly wrong results is the worst. It reminds me of a project I once did. It involved producing reports from tens of thousands of records including summing some of the fields. I was constrained to work on Windows so I put the date into an SQL server database and used MS Access to produce very nice looking reports. I reviewed the reports and everything "looked OK" so I handed them to the users for approval. They users were accountants. They added up the partial sums and pointed out that the results were only approximately correct. It turns out that MS Access is not so good at arithmetic. I restructured the reports to perform the arithmetic in the SQL queries and just use MS Access to format it for a pretty page. I also checked the arithmetic before handing the next revision over.
Better testing (as in more than a superficial glance) would have caught this before review but there always exists the possibility that subtle bugs can sneak past even well thought tests.
Agreed. I was doing an derailment investigation a number of years ago which involved digging through event recorder logfiles. The event recorder is such that it writes an entry every second but only updates GPS 20 seconds, so it writes GPS coordinates against every 20th entry.
What I discovered was that the event recorders on certain locomotives updated GPS at the 20th second rather than the 0th second. This meant that the GPS entry next to each line was in fact offset by 20 seconds - i.e. the entry for 02:11:40 was in fact what was sampled at 02:11:20. I think they must've held the GPS coordinate in memory somewhere but updated it AFTER writing that second's entry, so they wrote 02:11:20 whilst holding 02:11:00's GPS, then updated it, but then written that update at 02:11:40, etc. This was a fault with the design of the event recorders, not just one loco, as it occurred on each of that type that I looked at.
This confused me so much because it looked right - it was in the right general location, it was updating, etc. - but for a solid few days I did a bunch of analysis thinking the train was in a different location to where it really was. I eventaully picked up on it when subtle things kept not adding up and verified it by watching another loco come to a stop but then see the GPS keep moving for a bit afterwards until it settled.
I agree with you, subtly wrong results are the worst.
Better testing (as in more than a superficial glance) would have caught this before review but there always exists the possibility that subtle bugs can sneak past even well thought tests.
Just my own experience and opinion.