> It's a shame that pretrained approach leads to such good enough result. The learning-from-experience, or what should be the "right" approach, will stagnate.

We’ll see. I’m skeptical that you’ll ever get novel theories like special and general relativity out of LLMs. For stuff like that I suspect you need the interactive learning approach, and perhaps more importantly, the ability to reject the current best theories and invent a replacement.