I think the list of programmers throughout history who could reliably write something as complex as TeX using nothing but paper and pencil is incredibly short.
It's a lot easier than you think. When I was a kid I wrote longer programs in Basic using paper and pencil, because it was a lot easier for me than typing. I would type in the program after I was pretty sure it was right. The errors were almost always local. It seemed so easy to check the high-level structure of something when I could spread it out on the ground in front of me. Later, in college, I always printed out drafts of my papers and marked them up completely before I started editing.
"So Knuth too disagrees with the notion that unit testing always makes you go faster. Maybe he too is living in the stone age."
This follows him describing writing a program, in pencil, in 1977.