What does this say about the Chinese room argument?
Is the model a Chinese room or does it understand the game. If it's just a Chinese room, how come it is so effective, if it understands the game how can it be possible with just a rule machine?
It's the Chinese room but it is effective because the game space is relatively small (compared to the real world) and you can cover that space with RL guiding NLP language generation.
Let's make an analogy: the room is the human body. Proteins are the agent in the room. Chemistry is the rule book. Clearly the agent doesn't understand human concepts, but the system as a whole is.
Is the model a Chinese room or does it understand the game. If it's just a Chinese room, how come it is so effective, if it understands the game how can it be possible with just a rule machine?