Four justices are now older than Souter was when he retired at age 69. Thomas and Alito, 75, Sotomayor and Roberts, 70. Souter appeared to be in good health at retirement. I admire him for not clinging to power.
Not that I necessarily agree with the OP’s opinion, but who cares about fair when it comes to running the government? It should be about being effective.