> It is also a perfect example of how software is used to create increasingly more elaborate and faceless bureaucracies that force individuals to spend more and more time contending with them.
You are attacking the wrong target. It's the government that's broken. This kind of outrage can happen just as easily with pencil and paper. The root cause is the lack of accountability and desire to make the government function better.
But they can be undone with pencil and paper too. The footgun of automation here can only be undone with either a really good patch (git commit -m 'finally finally works for real this time') or a lot of pencil and paper work that's slower than the processes that caused the problem.
I'll note that this isn't the first time that people have said "well its the algorithm" when they were responsible. The example that springs to mind is bail risk assessments. You're very correct in that there are people making real decisions that are very cruel here. The machines give them something to hide behind.
Except software allows a scale and efficiency that is impossible with pencil and paper while also creating an ideal scapegoat. Software is being used to avoid accountability at a scale much greater than what was possible with manual process.
Isn't this the basis of the book "IBM and the Holocaust"[1] where the author lays out how IBM's technology helped facilitate the wide scale of Nazi genocide?
How is this a government problem? People frequently lose access to social network accounts and email because of broken algorithms. Google can blacklist a business and send it broke. Insurance companies, credit bureaus and banks can make a wrong decision and deny credit.
People decide to fix problems in software or anything else for that matter. If they don't get fixed it's usually a matter of somebody (or bodies) decided it was not a priority.
You are attacking the wrong target. It's the government that's broken. This kind of outrage can happen just as easily with pencil and paper. The root cause is the lack of accountability and desire to make the government function better.