>what is the LLM doing differently when it generates tokens that are "wrong" compared to when the tokens are "right"? If there is a difference, where does that exist? In the mechanism of the LLM, or in your mind?
If there were a detectable difference within the mechanism, the problem of hallucinations would be easy to fix. There may be ways to analyze logits to find patterns of uncertainty characteristics related to hallucinations. Perhaps deeper introspection of weights might turn up patterns.
The difference isn't really in your mind either. The difference is simply that the one answer correlates with reality and the other does not.
The point of AI models is to generalize from the training data, that implicitly means generating output that it hasn't seen as input.
Perhaps the issue is not so much that it is generalizing/guessing but the degree to which making a guess is the right call is dependent on context.
If there were a detectable difference within the mechanism, the problem of hallucinations would be easy to fix. There may be ways to analyze logits to find patterns of uncertainty characteristics related to hallucinations. Perhaps deeper introspection of weights might turn up patterns.
The difference isn't really in your mind either. The difference is simply that the one answer correlates with reality and the other does not.
The point of AI models is to generalize from the training data, that implicitly means generating output that it hasn't seen as input.
Perhaps the issue is not so much that it is generalizing/guessing but the degree to which making a guess is the right call is dependent on context.