It's not at all obvious that you would leave. Not knowing the specifics, a lot of us would lean toward asking for a pay raise rather than be promoted to the level of our incompetence.
I just think that a promotion shouldn't automatically entail moving into management. At my company the engineer ladder and the management ladder are different (though you can transfer between them if you like). I can get promoted 5 more times (NOT gonna happen, I'm hoping for 2 in the next 5 years) before I get stalled out. And people that are 5 levels above me are effectively gods.