Now you're the bullshit machine. No one said that. We expect basic reliability/reproducibility. A $4 drugstore calculator has that to about a dozen 9s, every single time. These machines will give you a correct answer and walk it right back if you respond the "wrong" way. They're not just wrong a lot of the time, they simply have no idea even when they're right. Your strawman is of no value here.
Clearly generative AI can currently only be used when verification is easy. A good example is software. Not sure why you think that I claimed otherwise.
The hidden assumption here seems to be that the model needs to be perfect before it has utility.