I’m shocked that Rust only gave you a 2X improvement over Python (I’ve rewritten a handful of Python services to Go and typically see 100-1000X improvement). What is the bottleneck? Was scaling horizontally an option?
I suspect the reason it's not faster is one of the following:
1. backend DB at its query limit
2. this is basically my first rust program and I've done something stupid
3. ab/wrk aren't scaling properly either (I know, not likely, but worth considering) when I'm testing it
4. bandwidth/IO limits on the AWS instance types I'm running this on
There are other possibilities too I guess. I was mostly doing this as a means to learn Rust though, I'd only go live with it once I had decent tests and everything else.
actix/r2d2 on the Rust side (I haven't worked out bb8 yet), aiohttp/uvloop/asyncpg/gunicorn on the Python side (Postgres for the DB). This kinda _is_ the cache at present, it's a distillation of a lot of other data into a given form. Rust's async story re: databases still looks a little up-in-the-air right now from an outside perspective though, so I guess I'll wait and see how it pans out.
It's definitely a work in progress and I didn't expect this much interest in my offhand "I tried Rust even though I couldn't see myself using it for real, but..." comment.
Interestingly, I was able to get an amazing speed improvement without going completely to Rust.
I didn't want to port APM code as well, so I kept node.js/express for routing. Very simple middleware that immediately passed the request and body buffer to Rust for handling. Rust returned a buffer back to express for sending the response.
It's easily able to hit 100k RPM on a single instance before hitting a CPU bottleneck in the Rust code (validating an RSA signature). It only needs to handle 90k RPM total.
