From the article and throughout the comments here it seems Backblaze prefers cheaper drives over a few percentage points of reliability. It would be interesting to see some data showing the tradeoff, but I suspect it reveals too much of their operation. At first glance it appears you can get a drive with .9% failure rate (HGST 7K3000) for $127[1], and yet BB really likes the WD Red, which has a higher failure rate (3.2%) and cost[2].
What might shed light without revealing too much is information about where they source drives today (their sourcing coverage during the shortage was very cool!). I suspect they're finding some nice bulk discounts somewhere.
> What might shed light without revealing too much is information about where they source drives today
Backblaze employee here -> we are willing to buy from anybody, we have no loyalty. Lowest price (for a particular drive model) always wins. Once per month we ask about 20 common suppliers for their "best price". We have bought from "B&H Photo Video", NewEgg, Amazon, etc among others. We're always willing to add more possible vendors, but I think we drop you from the list if the vendor bid prices don't even come close for 3 months - that means you don't understand anything and you're wasting our time.
With the volume you must buy in, why not buy direct from the manufacturers? I imagine they could supply you with their OEM pricing and product packaging, which seems like it would save money? Or is it a strategic reason like not getting stuck with one manufacturer?
I asked that question last time BB employees were here on HN.. the answer was they just don't buy enough volume to qualify to buy direct from the manufacturers. It sounds like you must but enormous quantities before the manufacturers will give you the time.
Unfortunately that's still the case (Yev here, from Backblaze). Minimum orders are around 10,000 unites, and we're just not there yet. Thinking about starting a consortium though, so if anyone needs hard drives... ;-)
We currently work with Distributors. I was referring to the manufacturers themselves, like buying directly from Seagate/WD. We currently work with a few different distributors to get different types of drives.
BTW, I'm not sure B&H Photo Video deserves scare quotes.
If you've been doing video and/or photo stuff for a long time, you know them as a rock solid distributor: I've been buying video stuff from them since the middle '90s or so, camera stuff more recently (e.g. my first serious camera, vs. Amazon they had better selection with competitive pricing). Joel Spolsky was sufficiently impressed with this home town operation to do a fascinating write-up on them: http://www.inc.com/magazine/20090501/why-circuit-city-failed...
One other note on who to buy from: if you're just buying one or a few drives, Newegg has gotten really serious about packing. http://www.pregis.us/en-us/productsandservices/productsoluti... inflated padding inside a fitting cardboard box. I suspect this is about as good as the packaging Seagate requires to return a drive for warranty service. Don't know about B&H, but as of a couple of years ago Amazon had a horrible reputation for packing bare hard drives.
If systems are designed with the expectation that hardware can and will fail often, better reliability drives aren't worth the cost as long as cheaper drives are relatively comparable. In addition to cost savings, your system has better robustness when it is decoupled from hardware reliability.
For example, Google's Map Reduce paper has a section on fault tolerance that goes into detail about how they handle the issue of failing workers:
Deltaqueue just posted the prices and there is a $5.00 difference between the two. So you pay 4% more for 2.5% less annual failures. That sounds close enough to be worth paying more for less operational expense. Obviously, backblaze gets a better deal or they would be buying up the Hitachi drives instead.
The AFR is 2.3 percentage points less (0.9% vs 3.2%), which in this case means that a single unit of the inferior brand is 3.5 times more likely to die during a full year of use. I'd love to see their calculations that justifies buying non-Hitachi drives.
I think percentage points is the right metric here. Spending a lot of money to cut down the frequency of a rare occurrence doesn't make sense, even if you can cut it down by 100x.
I have experience with hundreds of T of data stores. My opinion is very high of Hitachi 1T and 3T Deskstars. The problem is that they are not generally available - there could be months when you just could not order them.
Backblaze employee here -> Yes, this gets a SMALL amount of allowance. The datacenter team begs us to buy the Hitachi drives even at twice the price, but it would bankrupt us. But if the Hitachis are only $2 or $3 more expensive per drive (including the failure rate in that calculation) then we're willing to buy them for the reduced hassle.
I think the calculation is replacing one drive takes about 15 minutes of work. If we have 30,000 drives and 2 percent fail, it takes 150 hours to replace those. In other words, one employee for one month of 8 hour days. Getting the failure rate down to 1 percent means you save 2 weeks of employee salary - maybe $5,000 total? The 30,000 drives costs you $4 million, so who cares about $5k here or there?
> I think the calculation is replacing one drive takes about 15 minutes of work.
Is that really the true cost of replacement? I would think there is also the cost of dealing with the warranty and the testing and monitoring. Here is the quote from the blog post about unsuitable drives:
> When one drive goes bad, it takes a lot of work to get the RAID back on-line if the whole RAID is made up of unreliable drives. It’s just not worth the trouble.
I don't have the time to think about this fully, but it seems similar to calculating the present value of a future cash flow, because there are other costs beyond the first replacement effort:
> Their average age shows 0.8 years, but since these are warranty replacements, we believe that they are refurbished drives that were returned by other customers and erased, so they already had some usage when we got them.
It sounds like the total cost of a failed drive is actually 1.5x, because 50% of the replacement drives also fail.
The price we pay as consumers isn't going to be the same as that paid by Backblaze when they buy 100 drives at a time. The fact that they pick the WD Red implies that they're getting a better deal on those, otherwise they wouldn't settle for the higher failure rates.
EDIT: comments from BB employees below actually say they often buy off the shelf from NewEgg and Amazon.
It's a darn shame. We would love to pay less for hard drives. Even at our current capacity, we're still a "small fish". And since we try to run lean we likely could buy 10,000 drives and keep inventory, but that doesn't necessarily work out for us in the long run as hard drive prices tend to fall monthly.
Yea, unfortunately we aren't large enough to work directly with the manufacturers. We do buy from distributors sometimes, but if we can get a cheaper deal online, we'll go that route every time.
Psychoacoustics are still very much apart of the audiophile world; although, the internet has helped reduce the hype to some degree. People believe whatever they perceive and aren't inherently objective.
A/B tests do crop up from time to time in various audio / video communities (AVS and some car audio forums hold some pretty objective events) but science doesn't always provide an answer people want to accept.
It was interesting that there was a preference for the higher priced cables. It is too bad that I did not save my votes to see if my poorly trained ears matched what the others preferred. It is hard to say if there is justifiable benefit in spending big money for cables but I would say it is worth spending a little bit more than the cost of lamp cord based on this test.
With the digital cables to me it seems like a high data rate, low error rate and decent buffering system is all that is necessary for good sound. Given the data rates required for audio (even SACD is in the low MiB/s range) a cheap USB cable should do the trick.
Speaker cables are analog so they're going to have some impact on sound, and the study mentions a measurable difference in capacitance (the highest capacitance cables ranked best, the lowest worst — although the middle two are flipped).
It makes sense if there was an audible difference with speaker cables, particularly if the amps were high power and the cables represented a range of gauges. Trying to power a line array with 22 gauge speaker wire isn't going to work very well.
The stuff in that article makes monster cable look like a bargain.
No, but you can do some pretty straightforward electrical engineering math to determine that 22-guage speaker wire will fail in that case. Show me the physics and math that show failure with a USB cable. One of the major issues here is that USB is like the AK47 of the interconnect world, designed to work it some pretty terrible places. So when you experience jitter, it not actually the GD cable, it's most likely one of the devices on either side using a shitty controller and not handling data loss gracefully.
Audio is so subjective anyway. You'll never win an argument with these people. You just have walk past them and try not to laugh. It's the same as the issue on wines. Doesn't matter how many double-blind tastings you do, some assholes will just sleep better knowing they're not drinking shit I made in my bathtub, even if it tastes the same a Bogle sauvignon blanc.
Coinbase handled the recent surge in traffic pretty poorly. When prices tanked / fluctuated heavily, they turned off the ability to buy citing "a lack of bitcoin" and told users they could put in buy orders now for the market price on Friday or Monday. Two days later when the price is back up to ~$700 you can mysteriously buy again.
I'm not one for conspiracy theories, but USD liquidity has been extraordinarily frustrating over the past several days since most exchanges are foreign (and US bank wires take days to weeks).
I thought we were talking about converting to cash, not buying. I agree that the hard stop on buying is extremely annoying, and that makes coinbase a bad choice for playing the market. If you want to do fast transactional orders, CampBx is probably a better choice.
Can any artists on HN comment on how useful styli are with tips this thick? I suspect the added thickness is required for capacitive touch screens, but I always thought my fingertip was far too thick for accurately initiating thin lines or dots (e.g. in games like Draw Something). Something like this seems like it would require far less erasing:
For drawing, you can get away with larger tips, but I cannot understand how can people use those soft-rubber tips for capacitive screens, where the center-point of contact is actually moving as you scrub the screen. For comparison, I use a Wacom Intuos 5 large, and I'm already noticing that the pen tilt affects the position of the cursor (a 45' angle moves the point up to 0.5mm in the direction of tilt).
I saw several people taking notes with those pens though, but the note-taking applications offer a large "writing" area where you can write text as big as 4cm in height. I honestly cannot write efficiently at that size.
The "Pencil" here also has such a large tip that clearly covers the area of contact even if you use it with the short side up. I assume you will be able to figure out the point of contact after a while, but I would have chosen a normal pen design, not an "artisan pencil" at all.
The added thickness is to compensate for the fact that the iPad touch screen has a reasonable distance between sensors. You cannot get pixel-perfect accuracy on the iPad touch screen, and a pointed stylus would make that extremely apparent. So instead the trend is for larger tips, to encourage users to bump up their line thicknesses and to obscure the lack of pixel-perfect accuracy. Think marker, not pen.
It's my understanding that the inaccuracy is not due to the distance between sensors - but rather that Apple/etc have configured the sensors to only respond to contact points over a certain size (to reduce interference from power supplies and other imperfections)
It's certainly plausible that touches that are small enough are ignored, although I don't know if that's actually true. But even if it is, that doesn't change what I said.
> Consumption device, not a creative device.
Do you honestly believe that? Are you just burying your head in the sand, or do you actually think that all of the people out there happily creating things on their iPads don't count for some reason?
> Do you honestly believe that? Are you just burying your head in the sand, or do you actually think that all of the people out there happily creating things on their iPads don't count for some reason?
People working around the significant restrictions preventing content creation are the exceptions that prove the rule, in my opinion. I'm glad they're able to do it, but the iPad certainly wasn't designed to enable it.
I have a couple thick-point styli. I hate 'em. I just got a fine-point Adonit Script, which is not without its problems (no pressure sensitivity, not much app support yet) but oh man I can actually DRAW with the damn thing instead of scrawling with a big crayon.
That said I'm still finding myself just reaching for a traditional sketchbook and a cheap pen when I want to do some quick drawing, and opening up the laptop to run Illustrator when I want to do Serious Work.
So far I've seen "you can't print this out!" (who cares?) and "some of the transitions could be smoother!"
This one is my favorite:
"This isn't a one-page form. It's not going to fit on my desk. I'm going to pass on this candidate."
Seriously people? You think this guy doesn't have a hard copy of his resume? Your goal is to hire a talented candidate. This exercise demonstrates talent, and was created for exposure. Not optimization of communication in the form of a one-page CV that will end up decomposing on some hiring manager's desk that doesn't have the decency to reply to candidates he or she will not be hiring.
Well the other positive about this resume other than it being fantastic is that you know for certain you don't want to work anywhere that passes on you because of your beautifully designed, creative resume.
My first thought would be to scoff at your criticism. Would the birds' travel patterns be any less impressive if they were resting for very short periods during these 200 days? The sensor measures acceleration, so the only potential confusion might be with light movement near ground / trees.
But according to another study, some migratory birds rest for only seconds at a time during their flights:
At 4-minute intervals over 200 days, you have 72,000 datapoints. There are 1,920,000 9-second intervals (avg nap period from other article) over 200 days, so given their data collection spans only 3.75% of this time there's a chance they missed one of these naps.
That got me curious, so I ran the numbers on the chances they'd miss all of the naps; assuming these birds take only one 9-second nap a week (28 naps total over 200 days) there's a 35% chance the researchers would have missed all of them ((1 - (28 / 1920000)) ^ 72,000), which is pretty reasonable. But that chance goes down to 8% for one nap every 3 days, and to 0.055% for one nap a day. I'd say maybe one 9-second nap every few days is the lower limit of what I'd find plausible (assuming this was run for only one bird).
EDIT: Ah, missed this in the article - they were three birds. So the chances they'd miss all naps for all three birds goes down to 0.055% for one nap every three days, and 4% for once a week. So 9 seconds once a week is barely believable, but if they can reproduce this with another couple of birds that becomes really unlikely.
Those thrushes don't land to just nap for 9 seconds and then take off. They sleep for 9 seconds during a longer rest period, because they have to stay alert. They can't fly during the day because of airborne predators.
"My first thought would be to scoff at your criticism. Would the birds' travel patterns be any less impressive if they were resting for very short periods during these 200 days?"
The criticism isn't aimed at the birds; it is aimed at the logic of the researchers. The critic claims their conclusion does not follow from their data.
I have almost hit several emergency vehicles during the day due to my car's sound dampening and my car stereo (which doesn't have to be loud to block out sirens). At night, this has never happened because I have never not seen the seizure-inducing lights. My sample size may be less than a hundred, but you're making an assumption that piercing sirens at night are the most effective way to alert the public of an emergency. And for those blind pedestrians, low-frequency (or even lower decibel) sirens are more than sufficient.
I live on the 9th floor of a condo building that unfortunately has single-pane windows. If I don't have an extremely loud fan blowing, I will be woken up by everything from emergency vehicles to motorcyclists. I realize this comes with the territory of living in a city, but if cosmopolitan cities like Geneva can incorporate effective noise pollution laws* then there's room to improve.
This was my initial thought given the dealership limitations in North Carolina, Virginia, Texas, Minnesota, and New York, but do you know for sure how out-of-state sales work?
I'm asking because the report seems to use registrations and not just sales figures to describe the numbers in California[1]. To my knowledge a registration only occurs in the state where you're going to use the vehicle, but I don't know if the process varies for these trouble states.
Yes, but this shouldn't come as a shocker to anyone that uses Facebook and Android. You can arrange to have contacts synced in Android so that you can see updates / photos / phone numbers merged with your Google contacts. NSA access notwithstanding, Facebook is in the business of making it easier to interact and socialize with people.
Alternatively, you can use the app "Tinfoil" for Facebook, which is essentially a locked-down web browser that provides a watered down Facebook experience.
If you want to maintain a high reputation for your MTA's deliverability, ignore this post. Attempting to send hundreds, thousands, or tens of thousands of malformed addresses to domains (some of which will be well-formed) will result in a higher spam score that will ultimately create more work for whoever is managing your mail platform.
What might shed light without revealing too much is information about where they source drives today (their sourcing coverage during the shortage was very cool!). I suspect they're finding some nice bulk discounts somewhere.
[1] http://www.amazon.com/Hitachi-Deskstar-7K3000-HDS723030ALA64... [2] http://www.amazon.com/WD-Red-NAS-Hard-Drive/dp/B008JJLW4M/ (both seem to be market consumer prices)