Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

One argument I've often heard is that since it's easier to scale your application compared to your database, pushing the calculations on your application is a way to get a better scalability in the long term. I wonder how true that is though.


I’ve seen applications go down hundreds of times from pulling entire tables locally to compute on them. I’ve seen maybe once where an application grew so successful that scaling the database became an extremely challenging topic.

And even in that case, I/O and contention were inevitably the problems. Not CPU.

I’ve similarly heard myths that you should be judicious when writing indexes because they can affect insertion performance. I’ve again seen hundreds of cases where under-indexing killed performance and zero where over-indexing caused problems.

99.9% of the time, you’re not the crazy special case. And if somehow you are, the solutions required are going to be nuanced and involve a ton of specific measurement. It’s widely unlikely you’ll accidentally avoid these problems through something like this.


That depends a lot on the workload, of course. But quite often the DB is not constrained on CPU, but rathen on IO or memory (or even networking), and calculations mostly consume CPU.

Moreover, calculations usually require just a small subset of columns in a table, and thus can use indexes efficiently, whereas grabbing all the columns to then filter them in application (because that's how many ORMs work, at least by default) becomes not only worse in terms of memory and networking, but also in terms of IO and CPU required on the DB side.

So overall I'm sceptical of that argument, unless there's clear proof from profiling that it's indeed the case.


Depends a lot. You need to consider the requirements more in detail, and especially if we are considering reads vs writes, and also what are the methods to write the data that needs scalable reads.




Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: