Exactly. I could from memory recite the main story beats of The Lord of the Rings, and probably even get to the detail of all major plot points and some minor ones, and maybe even some famous phrases.
An LLM unburdened by restraint could like produce page upon page of story nearly identical to the original.
Moreover humans learn and evolve their knowledge from other experiences other than books and others’ code.
As you said LLMs just remix something semi randomly according to a weighted graph with no underlying knowledge or understanding whatsoever.