I guess I phrased that wrong. Explicitly, DevOps costs are tiny in a startup, even if you do it all yourself with a bare metal server, and moving 0.5% from pot A to pot B makes no difference.
Some businesses would require huge up-front investments without the likes of AWS. DevOps costs might overwhelm you pretty quickly once stuff like compliance becomes a factor, for example.
Sometimes it's not about the technical issues, but documentation, process and qualifications. In B2B there's plenty of that and just the bus factor [1] alone might force a start-up into considering a cloud provider.
In the end it's not just shifting cost, it's also shifting risk and standards and that may or may not be a critical factor.
Edit: The move from CapEx to OpEx is not about savings, it's often about shifting the costs in your books.