Yeah, people make this anthropomorphization leap into artificial AI because the conversational interface is kind of human-like but forget that the weights are trained once & fixed forever. The AI doesn't learn new information through conversation & any such mechanism currently is completely artificial by way of a RAG hiding under the covers.
Are we not very close to lifting this restriction? Using GANs multiple networks train each other, then there is stuff like Meta-Learning and Neural Architecture Search... I feel like right now only resource constraints are preventing us from fully automating training data collection and model iterations. Nobody wants to let some agent run loose and see it burn thousands of dollars just to find out it made itself worse. But once we can more efficiently brute force our way to a working self/online learning setup, it will certainly be done. We already synthesize training data using other neural networks too.
Even in that case you end up with an AI that is teaching itself based on the cumulative sum of all conversations it has with all people in the world basically (& needing to remember it). That is very different from me learning from a conversation with one person and remembering that. And my impression is that we're nowhere near seeing this deployed in production.
Sure, if you cut down the power requirements by 3-4 orders of magnitude you might get personalized agents. Still, the architecture is very different - in modern AI there's a very specific split between training & inference and I'm not aware of anything on the horizon that looks more like online training (+ the split is useful for all sorts of reasons).
Anyway, my point still stands - it's anthropomorphization because AI doesn't work that way today.
You're right, I was assuming that once the unguided training/optimization methods become cheap enough to perform continuously (and maybe in parallel to inference) it would be indistinguishable from online learning. For true online learning we're still lacking a good base architecture (although Meta-Learning and NAS are exploring that angle).
