You use LLMs to _discover_ how to approach important problems. You don't necessarily need to use the output verbatim. Same as StackOverflow and Google.
When you employ your developers at $200K/yr you won't trust them to tell you the first one hundred digits of pi, but you'll trust them with your business logic, which is much more important and mission-critical to you.
The difference is that (hopefully) your employee is honest enough
to say "I do not know the first 100 digits of Pi offhand but I can find out"
An LLM will happily produce a string of 100 digits that might be the first 100 digits of Pi, might be some known sequence of 100 digits in Pi but not the first 100, or might be 100 random digits that have nothing to do with Pi
I was actually curious about this and chatgpt actually accurately and very slowly gave me the first 100 digits of pi one digit at a time. I have _no idea_ how that worked, it did not search, nor did it run code. As far as I can tell, it pulled it straight out of it's own model.
If I ask it to use python, it writes and executes the code _much_ more quickly, same if I ask it to search.
But use them to do more important things that require more precision and accuracy?
No thanks