I wish that I could share some of the internal cost analysis that was a big part of the decision process; I've dropped breadcrumbs here and there, but exposing the whole thing just isn't a possibility.
Physically, the Mac Pro itself is really densely constructed. Even with some empty space inside our Mac Pro chassis, the solution is effectively 1U per 2 GPUs. That's pretty dense, and it hits our power target for the current site design, so going denser would only lead to stranding space ahead of power (which leads to cost inefficiencies).
But, let's consider some hypothetical configs with list prices that I just looked up. Anyone can do this, and these are not reflective of my costs (you can always do better than list). In reality, I would do a lot more digging on the Linux side, but this is a reasonable config that is analogous in performance and fits into my server ecosystem.
I'm excluding costs that would exist either way: the rack itself, CDUs, top-of-rack switch, cabling, and integration labor are all identical or at least very similar. Density is very similar, so there's no appreciable difference in terms of amortized datacenter overhead.
Mac Pro config (4 systems in a 4U chassis):
- 4x Mac Pro ($4600)
- Intel E5-1650 v2
- 16GB RAM
- 256GB SSD
- 2 x D700
- Our custom chassis
Capex only: $0.70 per gflop
Linux config (4 systems in a 4U chassis):
- SuperMicro F627G2-FT+ ($4900)
- 4x Intel E5-2643 v2 - 1 CPU each ($1600)
- 8x 8GB DIMMs - 16GB each ($200)
- 8x 500GB 7200rpm (RAID1) HDD - 500GB RAID1 boot drive ($300)
- 8x AMD FirePro S9050 - dual GPU ($1650)
Capex only: $1.03/gflop
For comparison, I'll give EC2 pricing as well. It's a tad unfair, since we aren't including on-going maintenance and electricity for the Mac or Linux options -- but 3 years of power is also not nearly equal to the cost of a server. EC2's pricing becomes truly atrocious when you consider network costs -- there is simply no comparison between 95th percentile billing and per-byte billing.
The Linux config for sure offers many more hardware options and greater flexibility -- and it also requires us to rewrite our imaging stack that is working out pretty well for us and our customers.
I firmly believe that we've made a pragmatic and sensible choice for our image rendering platform today. imgix has a number of smart and talented people constantly evaluating and improving our platforms, and I'm confident we will keep making the right decisions in the future (regardless of how nicely the Mac Pro may photograph).
x0054: "You are fitting triangular shaped computers, wrapped into round cases, into square shaped boxes."
And place them horizontally. And without additional fans!
And surprisingly, if you read skuhn's answers here, for them it all still has sense, financially.
And also surprisingly, Apple says it's OK to use the Mac Pros horizontally:
https://support.apple.com/en-us/HT201379
Fascinating.