Not that I want to advocate for Kubernetes, but this is just a stepping stone for another blog post in a few years "How managed services in AWS got us fired!" detailing how when AWS changed their pricing/strategies the companies had no way out of their design choices.
The "vendor agnostic" approach was the right call to make at that point in time. Sure, every business is unique and some cases fit better for a hands-off (lets pay for the convenience of AWS taking care of managed offerings) approach. But, it is a fallacy to think there is no cost to pay for that decision.
The cost of operating your vendor agnostic infrastructure is replaced by your team now needing to learn the intricacies of AWS such as IAM, AWS's way of networking, backups etc. Those operational needs dont just go away, they just become "easier" and more defined as AWS's way of doing it.
As a consultant, one must know where to draw the line and recommend the appropriate route.
This is easy if all you use is EC2, as K8s is effectively just a layer over EC2. Once you start using other AWS features you're locked in anyway, and tbh, what's the point of using AWS if all you're using is EC2?
Choosing to be vendor agnostic is purposely choosing to need to implement every single part of a stack yourself, rather than the (usually considerably better) AWS offerings, just in case you may want to switch away in the future. It's a massive waste of money and time for nearly every business.
Unless you're in the business of providing an infrastructural-level service (like heroku), you'll ship faster and cheaper by using a single cloud and going all in.
This is why I was not claiming that nobody should use AWS and everybody should be aiming for being vendor agnostic. You're right, many a times, for smaller businesses the vendor lock-in with AWS should be the least of their worries and should just go on about their business. From my mind, business that have bursty needs, the ability to scale up/down the infrastructure on AWS could be quite beneficial.
However, once you start growing and have the scale where you have dedicated teams of engineers operating your infrastructure, at some point it makes (economical and operational) sense to own your infra. For example, Instead of 10 different teams each having their own MSK setups to get a kafka experience spinning up so many instances for that occasional need, you could have a centralised kafka cluster operated by a central entity within the team. Instead of every team spinning up their own EKS clusters, and all the cruft associated with making it production ready, the company could set one way of running production services (and have a plan to be able to replicate that outside of AWS should there be a future need to).
> K8s is effectively just a layer over EC2
Yes, but one that comes highly abstract and lures you into tooling such as ingress, secret mgmt, image registry and all kubernetes artifacts that you need to care about in addition to your service itself.
> what's the point of using AWS if all you're using is EC2?
Not needing to operate a data center, networking, worrying about needing to visit the server to change the failing disk, and largely - being able to scale up/down on demand without capital expenses.
If I read what I wrote, perhaps it sounds extreme and my grey beard agrees to the ideology more than most HN audience would.
> However, once you start growing and have the scale where you have dedicated teams of engineers operating your infrastructure, at some point it makes (economical and operational) sense to own your infra. For example, Instead of 10 different teams each having their own MSK setups to get a kafka experience spinning up so many instances for that occasional need, you could have a centralised kafka cluster operated by a central entity within the team. Instead of every team spinning up their own EKS clusters, and all the cruft associated with making it production ready, the company could set one way of running production services (and have a plan to be able to replicate that outside of AWS should there be a future need to).
If you have multiple teams spinning up MSK, then you have a different problem and running your own infra isn't going to make this easier. Same thing with EKS clusters. This is a problem with not having a team that manages your infrastructure platform.
Product engineers should own the software they write, should own deploying them (but not the infra that deploys them), should own the oncall (but not the observability infra), and should have access to relevant platforms they need.
Platform engineers should build the platform that platform engineers use. Usually this does not include the CI infrastructure, but in some cases it should.
SRE often owns the observability platform (but in some cases there's an observability team for this).
This is more the norm in large size companies, and usually those teams are split down into multiple other teams that own different parts of the infra that platform engineers depend on for their services. This is usually the case whether or not the company uses a cloud or runs their own infra.
When you run your own infra, you have to have expertise in all the things that you were previously using the cloud for. IMO running k8s yourself is asking for trouble, and EKS is saving you a lot of time and money. Running your own kafka is basically masochistic. Running your own database infrastructure is also extremely difficult, and you'll almost certainly have worse uptime than if you used Aurora (or DynamoDB).
Though you can save money running your own infra, you're going to have to hire people to run it, and you're going to have to plan further out, and will be less agile.
Yes, EC2 is dope. But so is DynamoDB, MSK, EKS, SQS, SNS, SES, IAM, KMS, etc. If all you use is EC2, then you need to manage the equivalent of a large number of services that are high quality and well run, and you'll need staff with expertise in running them well.
The "vendor agnostic" approach was the right call to make at that point in time. Sure, every business is unique and some cases fit better for a hands-off (lets pay for the convenience of AWS taking care of managed offerings) approach. But, it is a fallacy to think there is no cost to pay for that decision.
The cost of operating your vendor agnostic infrastructure is replaced by your team now needing to learn the intricacies of AWS such as IAM, AWS's way of networking, backups etc. Those operational needs dont just go away, they just become "easier" and more defined as AWS's way of doing it.
As a consultant, one must know where to draw the line and recommend the appropriate route.