Our startup has a webapp that acts as a UI for a machine learning application. We have two different types of heavy workloads (ML and something else). The workers for these run on Kubernetes, which makes them easy to scale (which we do a lot, automatically). The app itself doesn't run on Kubernetes yet (simple VM). It would be better if it did, though! We keep building a lot of functionality ourselves (e.g. deployment without downtime) that would be easier with K8s. I think what you really want to avoid is Microservices, not K8s. K8s gives you many advantages, such as the entire config in Git, containerized workloads, deploy management, etc., which sooner or later, you'll find yourself wanting. Depending on how you get started, the learning curve can be steep, and some of the tools are not mature but supposedly used by everyone (ArgoCD for deployment?).
What is it giving you over an ec2 auto scaling group or ECS/Fargate?
Both can scale as much as you like, your config can live as cdk/cloudformation/terraform code?
I'm not an expert on auto scaling groups, but we have used the Google Cloud equivalent for a time. The biggest issue for us was that deployment is not as easy. Can you update the software on the ec2 instance without turning it on, for example? With K8s, we can leave the deployment scaled to 0 and just patch the image of the deployment to perform a release while the workers are all shut down. Similarly, we don't have to write code to wait for the workers to finish their current job before we shut them down in order to be replaced by a newer version; this is all managed by K8s, and the configuration for it lives in Git.
As others have already pointed out, it is also important for us to remain independent from Google Cloud / AWS.
Apache Spark can use Kubernetes as a scheduler out the box. I don’t know if op is using Spark.
A lot of data tools are starting to target Kubernetes directly as a runtime so using them with GKE/EKS is a bit simpler as it’s officially supported, allows to run locally and on the cloud with no vendor lock in.
ECS in a scaling group works well if your app is stateless but as soon as you scale workers dynamically, do service discovery, orchestration, you end up building some of the features Kubernetes provides.
