This was pretty much my thought as well. The results of this cost-benefit analysis make me raise my eyebrow, and, same as you, I can only assume that OS X permeates the infrastructure from top to bottom, to an extent that makes pulling it out too painful to even contemplate. Image processing shaders of this type aren't that hard to write.
If you're worried about them not matching some piece of client software exactly (Quartz Composer, Photoshop, etc.), you still have options. And those options - e.g., webapp for previewing/something else/etc. - you'll probably want anyway, for the benefit of designers that don't run OS X.
(The filtering aspect of the system I find a little surprising anyway - the idea of an image-focussed client-aware DPI-aware CDN makes sense to me (and I like it!), but something that does your Photoshop filters in the cloud sounds less compelling. I would have expected people to prefer to do that locally, and upload the result. But... while I've worked with many artists and designers, I'm not one myself. So maybe they'll go for that. And/or maybe a lot of their customers take advantage of the fact that the processing appears to be free. And I'm prepared to contemplate the possibility, however unlikely, that they might know their customer base better than I do.)
Uploading pre-edited images takes time/resources, and in general a lot of our customers rely on us to do all of their image processing so that they don't have to.
Additionally, creating edited versions of images in advance presents two problems: 1) Any future site redesigns or edits must now be applied en masse to the existing images or risk older images not complying with the new scheme, and 2) Instead of only managing the one original source image in the origin, now we're talking about maintaining all of the different edited versions, which is very inefficient from a storage and image management perspective.
There are many advantages to applying all of the image transformations on-demand, rather than in advance. Keep in mind that we are not simply photo filters, but a full end-to-end image processing offering (which applies everything from simple photoshop edits like cropping, face detecting, color correction, watermarks, etc. to automatic content negotiation and automatic resizing/responsive design) that works on the fly; this means that our customers now can make batch edits to their entire corpus of images through a few simple code edits.
This can become extremely cost-effective, but also helps in reducing page weight significantly.
I work at Hemnet, a Swedish real estate site, where we currently have about 300 000 - 400 000 new images each week. We recently investigated the on-demand approach but found that JPEG decompression was the biggest time consumer in our use case of scaling fairly large images down to sizes appropriate for web and mobile. This made us decide to do scaling in advance to all our sizes which resulted in overall CPU time savings.
It would be interesting to hear how img.ix solves this, since you are arguing for the resources savings in the on demand approach.
It is true that the time to fetch, render, and deliver an image the first time it is requested can be a bit longer, because of all the processing we're doing in the case of that single request, but because we then cache that fetched image, all subsequent requests are delivered an order of magnitude faster (50th percentile 45ms).
The majority of the time taken in the first request is actually in fetching the image from the origin source, so once it's cached in our system it becomes a much faster operation: and of course delivering the cached image without it traversing our stack is even faster.
So yes, the initial request can take time, but all of the subsequent requests of that image are much faster than the alternative. And when you take into account that our service makes it possible to send the correctly sized image for the display (instead of loading a preset size and displaying it smaller), and optimal file types based on device and browser (webp for chrome, etc), load times/page weights on all of those requests are significantly improved.
In general, anyone who serves multiple requests for their images over time will see a marked improvement on page weight/speed, compared to rolling their own solution where they have preset image sizes and deliver jpeg only.
Look at the current state of Mackintoshes. People are having kernel panics and struggling to keep their machines running with current software. OS X moves pretty fast, possibly faster than Linux, and Apple builds it to support Mac hardware.... the teams who are porting hackintosh code have to support a lot more hardware variety and they have less resources than, say, linux.
Running mackintoshes in production makes no sense.
And I challenge the claim that you would save money.
Looking just at off the shelf costs of low end hardware does not tell you the TCO of serious machines that need to be running all the time.
To get comparable hardware quality to Apple products you have to spend more, generally, when going with "commodity" hardware.
The idea that Apple is expensive is a myth, born of two things-- people perpetuating it since 1980 (yes, 35 years this myth has been spread), with the vested interest of rationalizing their dislike of Apple... and the fact that Apple doesn't compete at the very low end.
In production, TCO is much more about reliability and other things than initial hardware cost.
> People are having kernel panics and struggling to keep their machines running with current software.
Not here. My Mac is at about 11 days of uptime and it's under constant use. At this moment, I can't say it's less reliable then my Linux machines.
In this specific case, however, I'd consider ditching the enclosure and ducting cold air through the internal chassis/heatsink. A Macpro is, essentially a heatsink with boards mounted on it and I'd just let the chassis do that part.
It's hardly the uptime of someone who's struggling with kernel panics and random crashes. Last power down was when I embarked on a trip. The previous one was a system update. In fact, I never saw a kernel panic with this machine.
> People are having kernel panics and struggling to keep their machines running with current software.
A guy I work with has been running several heavily used hackintosh servers without issue. They have been very reliable and he's happily converted existing Linux servers to hackintosh. He's been doing this for a while and knows exactly what hardware to use.
It is appealing in a way, but I also think it would put the business at legal risk in a way that is totally unconscionable. We cannot run a Hackintosh in production for a single moment, the long term ramifications could be immense.
The chassis we designed represents my attempt to re-house the Mac Pros in something more suitable for the datacenter.
It'd be interesting to see someone rip apart a Mac Pro and build an entire form-factor around its setup.
Don't get me wrong, what you guys have done is extremely beautiful in all ways, but I can't help to think that if someone wanted to do this with say a Mini... say you take a 1u rack, drill some new holes into it... hmm.
The Minis are incredibly simple machines inside, and their airflow is not great. You could pop their logic boards out and run them without the chassis. With some thought it wouldn't be too tough to significantly improve their airflow for this environment.
I do have an existing rack design that holds 64 of them (and other people have gone denser, with operational compromises I prefer not to make), so there's no great impetus on my side to rip them out of their enclosures.
Thanks for the detailed response. Really cool stuff you guys have going on.
What other companies utilize Apple hardware in this way at this kind of scale? While not "out of this world" in comparison to some of the big players who have tackled scaling, it's definitely significant considering Apple hardware.
I'm not aware of anyone who specifically uses Mac Pros (although I've heard some private rumblings). I suspect part of the issue is that the old form factor was not very rack-friendly, and people haven't gotten comfortable with the new form factor. Maybe this chassis design will help move this forward.
Mac Minis are a little more common than it might seem at first glance, particularly for use cases that some other people have outlined in their comments throughout this thread. Mozilla uses them to test Firefox builds on OS X for instance. I would imagine that places like Sauce Labs must have a Mac Mini farm to facilitate browser tests on OS X.
I'm not aware of any other service that operates in the same space as imgix that runs outside of EC2, so they definitely aren't using OS X there. I think in general there's a sort of disregard for the particular graphics processing benefits that OS X provides (as evidenced by some of the comments in the thread).
I would also be remiss to not mention Mac Mini Colo (http://macminicolo.net/) who do co-located hosting. imgix started out with them, and they did a great job.
There's another interesting use case where you need to have OS X (or iOS): when you want to display photos taken on iOS devices with their applied filters (the images are stored pristine, and the filters are applied on top when you view them). To recreate these photos exactly as they were on the device, you ideally need to render it within an Apple environment. You can probably imagine the use case for a service that stores a lot of user generated photography, in a world where iPhones are the most popular cameras (https://www.flickr.com/cameras).
I also heard through the grapevine today that a certain film studio is interested in getting one of these chassis to test out, because they saw this article. That's pretty exciting to hear, even though we don't profit in any way from the sale of these chassis.
If you're worried about them not matching some piece of client software exactly (Quartz Composer, Photoshop, etc.), you still have options. And those options - e.g., webapp for previewing/something else/etc. - you'll probably want anyway, for the benefit of designers that don't run OS X.
(The filtering aspect of the system I find a little surprising anyway - the idea of an image-focussed client-aware DPI-aware CDN makes sense to me (and I like it!), but something that does your Photoshop filters in the cloud sounds less compelling. I would have expected people to prefer to do that locally, and upload the result. But... while I've worked with many artists and designers, I'm not one myself. So maybe they'll go for that. And/or maybe a lot of their customers take advantage of the fact that the processing appears to be free. And I'm prepared to contemplate the possibility, however unlikely, that they might know their customer base better than I do.)