I'm fairly sure OP was more concerned about modern GPUs being used as TPUs or whatever they're called, than about what graphics circuits the Atari 2600 was using.
Even mid range GPUs are proportionally much more expensive. I built a decent gaming PC with a GTX 760 10 years ago for about $900. These days you'd have to pay double for the same relative performance.
This seems commonly believed but isn't really accurate at all. For instance here [1] is a build to play Cyberpunk 2077 for less than $450 (low, 1080p, 50+ FPS). And that game is clunky, and that build is valuing price > performance. Add a couple hundred more and it'll run that game (or any other) like a beast.
Also, especially in modern times, keep in mind how real inflation has been. That $900 PC from 10 years ago would be $1200+ today.
Again, there are plenty of games to chose from besides last generation AAA games.
I guess some folks might suffer from FOMO, but that doesn't change the fact there are more games to play than most folks can achieve to finish during their lifetime, that aren't last generation AAA.