But governments do slash budgets or cut programs altogether. And agencies typically continue on because their existence is mandated by law and (unlike the private sector) they can't choose to just ditch those customers that cost a lot to deal with. There are big variations in administrative competence to be sure, but the logical conclusion of your argument is that if you hate the President (or rather, the administration of the federal government by a given President) then the logical thing to do is get rid of the government.
I don't understand how " the logical conclusion of your argument is that if you hate the President... then the logical thing to do is get rid of the government." The voice vs exit paradigm has been employed in some arguments for anarchism, and many arguments for limiting the scope of government, but those arguments have nothing to do with the executive, and I didn't make any such argument.
I know you didn't, but where is the line between saying individual agencies are inefficient and should be shut down and the administration as a whole? There's certainly a contingent (not necessarily including you) that wants to scale the federal government back to the bare minimum sketched out in the Constitution and devolve everything except national defense and certain enumerated powers back to the States.
There is no line to draw, but there is a counter-argument. The (most effective) counter-argument is that the government agencies are capable of achieving objectives that the private organizations can or will not.