People often easily conflat religion, theology and theocracy. But an atheist dictator that would put death sentence for anyone that engages in a religion would obviously not be more ethical than some religious zealtoth doing the same for atheism. I'm not aware of any politically powerful atheist in history that went this path. I mean, most likely Stalin was atheist, but I doubt it ever was a matter to send people to gulag. Atheism is just rare as an ontological belief, so it makes sense most faith intolerance happened between religious dévots.
Note that strictly speaking, atheism is just rejecting the existence of any god. That doesn't necessarily make magic out of the equation. Though certainly atheism is generally associated to the rejection of any superstitious belief.
But just believing that ZFC make a sound mathematical foundation doesn't make you an advanced flawless logician. People stay mere humans, whatever they believe might be the most relevant foundation to use their more or less weak reasoning ability.
Mathematics is seen a formally grown, logical system, that has features that are "discovered" rather than invented .. "Given some {X}, {Y} follows without question".
It is understood that one can tweak an axiom, the fifth posulate for example, and get a different logical ediface - a non Euclidean hyperbolic geometry in that case.
The ZFC "Axiom of Choice" has bearing on infinities and other things, including many proofs that depend on reduction by absurdity.
Note that strictly speaking, atheism is just rejecting the existence of any god. That doesn't necessarily make magic out of the equation. Though certainly atheism is generally associated to the rejection of any superstitious belief.
But just believing that ZFC make a sound mathematical foundation doesn't make you an advanced flawless logician. People stay mere humans, whatever they believe might be the most relevant foundation to use their more or less weak reasoning ability.