You seriously underestimate what a process that's "generating new word combinations that look like the word combinations its already seen" can do, even when air-gapped (which ChatGPT isn't). Right now, at this moment, people are building closed loops based on ChatGPT, or looping in humans which are seriously intellectually underequipped to deal with plausible insane output in that quantity. And those humans operate: machinery, markets, educate or manage other humans etc etc.
To me, that's the real danger. ChatGPT convincing a human something is true when it isn't. Machinery is a good example, maybe ChatGPT hallucinates the safety procedure and someone gets hurt by following the response.
