Not sure why this is popping up years later, but mrshoe and I presented this at PyCon in 2007. Obviously, the landscape of individual technologies has changed since then, but the basic approaches are still true and pretty language-agnostic (it was just natural at a Python conference to have the talk in Python-centric terms).
Under what circumstances is storing static media in memcache and using a lighttpd/nginx module to serve them going to be faster than using the web servers' built-in sendfile support? Sendfile is a DMA operation, and as long as the disk cache is warm (same odds as having enough RAM for memcache to keep all media, I think), it won't hit the disk after the first time the media is requested. Is reading from memcache and then sending to the client ever really better than that?
Uh, iirc, the assumption here was that the proxy and the application server (and its resources) were not necessarily located on the same physical machine. Serving static content off the proxy is certainly another option, but it involves a different kind of deployment than having the application servers still provide the (initial) request for static content.
In my environment we've also observed latencies associated with some of the appservers referenced, so we rolled our own thin one with low-overhead in mind. We use multiprocessing as our means to saturate multiple cores, and avoid the nuisances associated with general-purpose multithreading.
The deck covers load-balancing but doesn't cover other patterns, such as the async variety. We increasingly use queues of various sorts. Zmq & mongrel2 offer really exciting possibilities, though we haven't tried the latter at scale so far.
The points are probably applicable to most scaling discussions, and not particular to python. I'm surprised there isn't any discussion of using a CDN. Its easy to eliminate static load, and it reduces latencies considerably.
I do wish he went more into detail about DB optimizations, since this is usually where most people struggle to add performance to their applications.
This is a pretty general high-level overview on how to scale. Nothing really Python specific besides the web servers. I thought the how to scale Ruby tutorials on New Relic were much more useful and in depth.
