Government is controlled by the highest bidder. I think we should be prepared to do this ourselves by refusing to accept money made by unsafe businesses, even if it means saying goodbye to the convenience of fungible money.
Banding together and refusing to accept harmful money is indeed akin to creating a government, and would indeed be more effective at controlling people's behavior.
But participation would be voluntary, and the restriction of harmful behavior would apply to it's enemies, not its citizens. So I'm not quite sure what the problem is.
Replace government with collective society assurance that no one cheats so we aren’t all doomed. Otherwise, someone will do it, and we all will have to bear the consequences.
If only enough individuals are willing to buy these services, then again we all will bear the consequences. There is no way out of this where libertarian ideals can be used to come to a safe result. What makes this even a more wicked problem is that decisions made in other countries will affect us all as well, we can’t isolate ourselves from AI policies made in China for example.
