It is, and as an amateur player I find it a very frustrating rule to play by.

Wouldn't it be even more frustrating if your opponent was constantly picking up pieces when thinking?

I can't say I've ever had that problem while playing chess (even without the rule), but if I did, I'd ask them to try doing it less.

It's a perfect example, that rule is not there to make it easier to play the game, it's there to make it harder.

This is common in lots of sports, think about pistol shooting, you're supposed to shoot one handed, arm straight, shoulder to target. That's NOT the best way to shoot if your goal is to hit the target, it's the best way to shoot if you want a challenge.

That's what makes the example good as you might remember the pain from this rule being enforced when you've touched n mistake

Yes, but you can at least move it elsewhere as long as you haven't let go.