Thanks, I specifically looked to see if this sort of comment was here. Not finding one would mean that there's a fair chance that there's some well-founded with plausible details sci-fi writing.

I believe gwern deliberately did not go for nanobots because they're kind of unrealistic, or at least have a reputation for being implausible, on a purely physical level. As such, centering AI risk around nanobots would have taken away from the actual threat, rampant intelligence.

I'm wondering why you believe this? Especially when you yourself are a collection of nano machines with much harder design constraints than the artificial variety.

That's precisely why. While there's a lot of room for design improvement in complex systems, it seems likely, or at least more arguable, that nature has largely cleared out the low-hanging fruit for single-celled organisms.

All of engineering is a counter example. We have a fiendishly hard time replicating the precise lift mechanism of flying birds (this is still an active are or research), yet we have designed and built the Boeing 787 and SR-71.

Life is an existence proof that atomically precise nano machines are possible, but it is not in any way a demonstration of their limitations any more than birds represent the epitome of heavier than air flight.

Correction: I read up on the story to refresh my memory, and Clippy does use a form of nanobots. I was mixing it up with another story, that just used a timed hyperlethal pathogen.