> the default storage options did not support ReadWriteMany for Block and Object Storage, which were necessary for using my database Postgres.
Postgres does not require ReadWriteMany volumes.
> To better understand this, when I created multiple replicas, the same database could not be accessed by all the replicas. Only one at a time could access the database.
Assuming you mean multiple replicas accessing a single postgres database, this should not be a problem. I can't think of any networked database that prevents multiple clients from concurrently connecting. Postgres even has MVCC to allow multiple concurrent changes to the same record.
The architecture diagram towards the end of your blog post shows multiple subdomains, each with their own database. Each of these should be using their own RWO volume, not a single ReadWriteMany volume; that's a good way to cripple your ability to scale.
CNPG is my preferred way to manage postgres clusters on Kubernetes.
k8s supports stateful data just fine. The advice should be "keep your data out of dbs where clustering is an afterthought (if a thought at all) in the year 2024"
Obviously title is clickbait - OP basically deployed a whole set of pods including a db and api layer, basically the entire stack, for each new customer. This is difficulty level "nightmare" and not how most people probably use k8s.
This design has its place and I've built variation of this before, the challenges of doing it this way have nothing to do with k8s IMO. In fact k8s makes it considerably easier to implement which I think is true challenge here - it gives you just enough rope to hang yourself.
Yes, but in a large enterprise setting, which is where it makes most sense from a cost standpoint. For smaller business development, I typically go with something like fly.io.
Postgres does not require ReadWriteMany volumes.
> To better understand this, when I created multiple replicas, the same database could not be accessed by all the replicas. Only one at a time could access the database.
Assuming you mean multiple replicas accessing a single postgres database, this should not be a problem. I can't think of any networked database that prevents multiple clients from concurrently connecting. Postgres even has MVCC to allow multiple concurrent changes to the same record.
The architecture diagram towards the end of your blog post shows multiple subdomains, each with their own database. Each of these should be using their own RWO volume, not a single ReadWriteMany volume; that's a good way to cripple your ability to scale.
CNPG is my preferred way to manage postgres clusters on Kubernetes.