According to [1], they trained an LLM on legal Othello moves, and 1) it got an error rate of 0.01% 2) when they analyzed its network, they found a model of an Othello board 3) when they modified the in-network model, it started generating moves legal in the modified board position.
In other words, the LLM did build an internal model that contained the rules of Othello merely from seeing legal moves. It's reasonable to assume that the same thing is happening (at least to some degree) with LLMs based on human speech.
In other words, the LLM did build an internal model that contained the rules of Othello merely from seeing legal moves. It's reasonable to assume that the same thing is happening (at least to some degree) with LLMs based on human speech.
[1] https://thegradient.pub/othello/