Crypto code and non-crypto code have different requirements. For compiling non-crypto code you often want the fastest code that still matches the language spec. "Fast" is usually even relative to target platform. For crypto code, you need a very narrowly defined abstract machine that you program against and since that's a much stronger requirement than what the language is defined for, you take the next best thing by turning off all optimizations and hoping you understand well enough how your compiler generates code.

In other words, they already do their best by giving authors control over how much optimization is applied. Anything beyond that is a language design problem, not compiler construction problem.