Not clicking "yes" would work if most sites weren't, in fact, sending all tracking/setting all cookies by default, which most are :/

At least it doesn't give them a legal basis for doing so, while clicking "yes" does.

In practice, the real defense is a good ad blocker which would block both the consent prompt and the associated trackers.

Same here, as previously learned on HN :)