It is not obvious to me that this is true.
The argument against this is that a model has to develop an abstract understanding of the world in order to get better at predicting the next token.
You'll find different opinions as to wether this happens with todays llm's, but it's definitely not a clear cut issue.
I think they only develop abstract understanding about the relationships of the words/group of words.
It is not obvious to me that this is true.
The argument against this is that a model has to develop an abstract understanding of the world in order to get better at predicting the next token.
You'll find different opinions as to wether this happens with todays llm's, but it's definitely not a clear cut issue.