Eh, I have had this issue in racing VR games (with a racing wheel setup) with shifters on the wheel. I was constantly confusing which was which as the wheel was spun around. I think it's something you just get used to.
Stuff that moves with the wheel is horrible. I’m in a turn, and I need to adjust anything (in my car, that’s phone and paddle shifting) and I need to chase them around in weird places. “Getting used to things getting harder” is a step back, not a premium experience worth paying those bucks for.
BMW has a nice fix for this. Paddle on the rear always shifts up, then there’s a paddle you press with your thumb that downshifts. Same thing on both sides you can downshift while turning the wheel by using your thumb on the top “paddle” instead of remembering which is left and which is right.
The shift controls on my 7th gen Celica are similar, button either side on front of the steering wheel always shifts down, press with the thumb; button on the back of the wheel always shifts up, press with index finger. This car is 20 years old.
The Celica has the cruise control stalk turn with the wheel, which is okay. My VW T5 Transporter's cruise control stalk is stationary affixed to the steering column, and that's okay too.
I also had a Honda Jazz that had paddles like you described.
I haven't tried many newer vehicles, but I can't see how anything they offer is much of an improvement.
