And that program would be written different each time and sometimes fail.

...and the LLM, given an agentic loop, would ingest its own error message and correct itself...

...and eventually it'd persist some knowledge in a context window to not make that mistake for a while...

...and then it'd forget and make the same mistake again...

exactly