I've thought of the same analogy, but you know, I've never actually seen someone go "It's wrong about the stuff I understand, but I'll trust it anyway on everything I know nothing about". It's either:

(1) people getting caught using it to do their own jobs for them (i.e. they don't even realise it's wrong about the stuff they do understand);

(2) people who see the problems and therefore don't trust them anywhere at all (i.e. no amnesia, quite sensible reaction);

(3) people who see the problems and therefore limit their use to domains where the answers can be verified (I do this).

--

As an aside, I'm a little worried that I keep spotting turns of phrase that I associate with LLMs, for example where you write "you're absolutely right": I have no idea if that's all just us monkeys copying what we see around us (something we absolutely do), or if you're using that phrase deliberately because of the associations.

The only thing I'm confident of is that you're not doing is karma-farming with an LLM, but that's based on your other comments not sounding at all like LLMs so why would you (oh how surprising it was when I was first accused of being an LLM), but eh, dead internet theory feels more and more real…

> The Gell-Mann Amnesia effect. And you're absolutely right, it's extremely pronounced in LLM users.

And I guess a lot of LLM-hype critics have the trait to be much less capable of "being able to flip to the next page and read a story on a subject they're not an expert on and they just accept all of it without question".

Because this is an unusual personality trait, these LLM-hype critics get reprimanded all the time by the "mob" that they don't see the great opportunities that LLMs could bring, even though the LLMs may not be perfect.