> Why is kube and conventional hardware mutually exclusive?
Conventional setup might've been a better choice of words. That being said, in my experience, k8s in smaller setups tends to use more small machines, rather than (very) few big ones.
> It’s just a different way to utilize resources. I don't see an issue with the costs - [...]
Assuming the developer knows k8s and the app doesn't need much integration [0]. If one of those isn't met, you'll need to invest the time to rethink you app and infrastructure. And you don't really want to setup such a setup with k8s knowledge gathered from a few medium posts, so you do really need to hire an expert, plus plan some more hardware for your controllers - there's definitely a money issue.
> [...] I see an issue with this massive list of inventory that needs to be carefully managed.
> Managing 30 boxes is not easy when they are all funny shapes and sizes. Are they all running the same OS? Same version? How do you roll out security updates? With kube, you just bring a new node online and then kill the old one. Done.
Going by the price, those are most likely dedicated boxes. So it's not like spinning down an EC2 machine and starting a new one (and not going for dedicated with these hw requirements would increase costs quite a bit); instead, you'd need to reimage. So, basically, you still have to manage those boxes - in addition to also updating your containers, now.
Or you simply enter `apt install unattended-upgrades` and move on with your live. Managing 30 hosts is really not that big of an issue nowadays.
> My argument still remains - what happens when a vital box dies? How quickly can a new one be brought online? Is that automated?
For the fronted, you could only really solve that by running a second box in the same size. And if you did that, you could automate that quite easily with something like a heartbeat - a failover can be handled without k8s. And do you really need all that uptime? This is a non-profit, not a startup; being down for an hour does not end your business.
> What about when the ES boxes shit the bed and you realize you actually needed a cluster of 5 nodes? Do you buy 5 new boxes that are carefully sized, or do you just scale your mega cluster from 4 to 5 and add a few new pods?
I give you the point, for ES it would really be useful. But horizontally scaling a cluster is really not that hard.
> Kube is not nearly as big and scary as this community makes it out to be.
No, it really isn't. But it does bring in a lot of complexity - how do you handle ingress? Storage? Audits? How do you scale? Does your app run in pods, can it easily fail-over? How do you handle shared state? You also need some additional hardware to handle controlling etc. and, as mentioned above, the knowledge to configure and run a cluster for such a high-profile site.
That being said, I agree with you that, seen from a clean-state, using k8s (or maybe just k3s) here would not be a wrong decision. For all we know, they might actually be using it for ES or some other parts. But we shouldn't disregard that k8s also brings in a lot of complexity, configuration overhead, costs and, next to that, a whole paradigm shift - your storage goes from permanent to ephemeral, unless explicitly marked otherwise, and accessing your app works completely different, just to name a few things. Learning k8s takes time and, while it is definitely worth it (IMO), you can still manage an infrastructure of that size with easier tools without any problems whatsoever. It's basically sure that the overhead in both learning and operations is simply not worth it and would cost much more than any savings k8s brings - which is exactly what my original response pointed out.
[0] The app might depend on specific OS features, use a lot of storage (in which case you will also need to setup a properly HA storage provider) or have some significant state, which would need to be synced to a new instance.
Conventional setup might've been a better choice of words. That being said, in my experience, k8s in smaller setups tends to use more small machines, rather than (very) few big ones.
> It’s just a different way to utilize resources. I don't see an issue with the costs - [...]
Assuming the developer knows k8s and the app doesn't need much integration [0]. If one of those isn't met, you'll need to invest the time to rethink you app and infrastructure. And you don't really want to setup such a setup with k8s knowledge gathered from a few medium posts, so you do really need to hire an expert, plus plan some more hardware for your controllers - there's definitely a money issue.
> [...] I see an issue with this massive list of inventory that needs to be carefully managed. > Managing 30 boxes is not easy when they are all funny shapes and sizes. Are they all running the same OS? Same version? How do you roll out security updates? With kube, you just bring a new node online and then kill the old one. Done.
Going by the price, those are most likely dedicated boxes. So it's not like spinning down an EC2 machine and starting a new one (and not going for dedicated with these hw requirements would increase costs quite a bit); instead, you'd need to reimage. So, basically, you still have to manage those boxes - in addition to also updating your containers, now.
Or you simply enter `apt install unattended-upgrades` and move on with your live. Managing 30 hosts is really not that big of an issue nowadays.
> My argument still remains - what happens when a vital box dies? How quickly can a new one be brought online? Is that automated?
For the fronted, you could only really solve that by running a second box in the same size. And if you did that, you could automate that quite easily with something like a heartbeat - a failover can be handled without k8s. And do you really need all that uptime? This is a non-profit, not a startup; being down for an hour does not end your business.
> What about when the ES boxes shit the bed and you realize you actually needed a cluster of 5 nodes? Do you buy 5 new boxes that are carefully sized, or do you just scale your mega cluster from 4 to 5 and add a few new pods?
I give you the point, for ES it would really be useful. But horizontally scaling a cluster is really not that hard.
> Kube is not nearly as big and scary as this community makes it out to be.
No, it really isn't. But it does bring in a lot of complexity - how do you handle ingress? Storage? Audits? How do you scale? Does your app run in pods, can it easily fail-over? How do you handle shared state? You also need some additional hardware to handle controlling etc. and, as mentioned above, the knowledge to configure and run a cluster for such a high-profile site.
That being said, I agree with you that, seen from a clean-state, using k8s (or maybe just k3s) here would not be a wrong decision. For all we know, they might actually be using it for ES or some other parts. But we shouldn't disregard that k8s also brings in a lot of complexity, configuration overhead, costs and, next to that, a whole paradigm shift - your storage goes from permanent to ephemeral, unless explicitly marked otherwise, and accessing your app works completely different, just to name a few things. Learning k8s takes time and, while it is definitely worth it (IMO), you can still manage an infrastructure of that size with easier tools without any problems whatsoever. It's basically sure that the overhead in both learning and operations is simply not worth it and would cost much more than any savings k8s brings - which is exactly what my original response pointed out.
[0] The app might depend on specific OS features, use a lot of storage (in which case you will also need to setup a properly HA storage provider) or have some significant state, which would need to be synced to a new instance.