I mean.. The LLMs are based on neural networks? They can approximate any equation right? It's possible the LLM formed areas that are extracting the equations and doing something very similar to traditional calculation. I think that "predicting" the next token is probably a bit reductive to what's happening in the LLM based on my understanding of image classification networks.

Humans don't work like binary computers either. Every person probably has some error rate on calculations as well. Maybe the LLM is much closer to humans in doing math than either to a physical processor.