This is why I love using the Deep-Seek chain of reason output ... I can actually go through and read what it's 'thinking' to validate whether it's basing its solution on valid facts / assumptions. Either way thanks for all of your valuable write-ups on these models I really appreciate them Simon!

Nota bene - there is a fair amount of research that indicates models outputs and ‘thoughts’ do not necessarily align with their chain of reasoning output.

You can validate this pretty easily by asking some logic or coding questions: you will likely note that a final output is not necessarily the logical output of the end of the thinking; sometimes significantly orthogonal to it, or returning to reasoning in the middle.

All that to say - good idea to read it, but stay vigilant on outputs.

That's a good note. I use DeepSeek for early planning of a project because of how valuable its reasoning output can be. It's common that I'll describe my problem and first draft architecture and see something in the output like "Since this has to be mobile optimized..." Then I'll stop generation, edit the original prompt to specify that I don't have to worry about mobile, and run it again.

I think is the right way to do it. Produce with LLM, debug and read every online. Delete lots of it.

Many people fear this approach for production, but it is reasonable compared to someone with a single course in Coursera writing production JS code.

Yet, we tend to say the LLM wrote this and that which implies model did all the work. In reality it should be understood as a complex heavy lifting machine which is expected to be operated by a very well prepared operator.

The fact I got a Kango and drilled some holes does not make me engineer right? And it takes an engineer to sign off the building even thought it was archicad doing the math.

Serious question: if you have to read every line of code in order to validate it in production, why not just write every line of code instead?

Because it's much, much faster to review a hundred lines of code than it is to write a hundred lines of code.

(I'm experienced at reading and reviewing code.)

This sounds like a recipe for destructive bugs and security vulnerabilities to slip into production.

Reviewing is really hard to do well. Like, on a psychological level. Your brain just starts nodding and humming along, pretending to understand. Humans have to consciously "perform review" to actually review. For example, https://en.wikipedia.org/wiki/Pointing_and_calling and checklists in aviation and health care, Tom Gilb's "Inspection" JPL-inspired spec review processes.

Even HN gets a steady drip of "look at my vibecoded project" -- "umm, you just leaked your API keys".

It's just that reviewing doesn't matter for a space invaders clone.

Reviewing isn't nearly as hard if you told the model exactly what to write already: https://simonwillison.net/2025/Mar/11/using-llms-for-code/#t...

Simon, don't you fear "atrophy" in your writing ability?

I think it will happen a bit, but I'm not worried about it.

My ability to write with a pen has suffered enormously now that I do most of my writing on a phone or laptop - but I'm writing way more.

I expect I'll become slower at writing code without an LLM, but the volume of (useful) code I produce will be worth the trade off.

Reading other people's (or llm's) code is one of the best ways of improving your own coding abilities. Lazy people using llms to avoid reading any code is called "vibe coding", and their abilities atrophy no matter who or what wrote the code they refuse to read.

Absolutely false for anything but the most braindead corporate CRUD code.

We hate reading code and will avoid the hassle every time, but that doesn't mean it is easy.