That's a great point that I haven't considered before. Back on the PS2 there was an "eyetoy" camera addon that allowed you to interact with certain games. "Eyetoy: kinetic" was a favorite of mine and my nieces/nephews: they could punch and kick and get rid of enemies or blocks or whatever on screen, instantly usable compared to the abstraction of a controller. It was easily the most engaging fitness I've done. It had its faults (proper spacing and lighting was critical), and now that you mention it I'd love to get into similar things in VR.