After looking at lots of these new models, I've come to the conclusion that they're all basically weighted gibberish generators that produce output not entirely dissimilar to old fashioned hidden markov chains -- just more sophisticated. The corpus is larger and the weighting scheme is much more sophisticated (i.e. you can use prompts), but at the end of the day they're sort of just barfing out nonsense that does a better job at tricking humans into thinking they're doing something smart. These models have really just introduced the concept of differing "qualities" of gibberish on a sliding scale from

   random words ---> markov models ---> transformer ---> human writer

Inevitably, users of these kinds of models want them to produce more and more specific output to the point that they really don't want what the models produce and instead are just trying to get a computer to write stuff for them that they want. Eventually all the tuning and filtering and whatnot turns into more work than just producing the output the user wants in the first place.

It's just a room of monkeys banging on typewriters at the end of the day.