LLM is a model. So, it fits under "all models are wrong, some are useful". Of course, it can produce wrong results. But it can also help with mechanistic tasks.

And you can run some models locally. What does he think of open-weight models - there is no source code to be published. Closest thing - the training data - needs so many resources to turn into weights that it's next to useless.