Sun engineer responds to the Backblaze "Petabytes on a budget" design • Hacker News Board

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#21

Post by notaddicted » Thu, Sep 03, 2009, 7:39 PM UTC

jacquesm wrote: post #5

[deleted] wrote: post #4

Earlier quoted context omitted.

[deleted]

> Given the reliability info he cites, the fact that backblaze is still in business must mean their usage-to-storage ratio is small. No, it probably means that they get their reliability at a different level, such as simply storing the data on a number of machines. Systems like glusterfs make this completely transparant.

( ahhh I deleted my post before I saw the response )

From the numbers he cited I got the impression that the drives they are using would fail 3+ times more often than enterprise drives (lower MTTF + speced at low temps). The cost of replacing the drives (even if they are 1/2 the price) would make the system uneconomical.

Also: because of their case design (non-hot-swappable) the cost of human replacement is higher too.

Thus they must have a low usage rate to have a low failure rate for their system to make business sense.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#22

Post by abyssknight » Thu, Sep 03, 2009, 7:40 PM UTC

smanek wrote: post #3

I think everyone agrees the Backblaze (bb) has no performance - and that's fine. But a lot of his complaints are about reliability, and thats kind of a moot point since Backblaze (and anyone sane) stores every piece of data on at least two (and ideally 3 or more) separate bbs. I would argue if you don't need performance (i.e., no huge database), but you do need a lot of space, you're better off just setting up two 'm…

I read that as "every piece of data on at least two (and ideally 3 or more) separate bbs", meaning BBS as in a bulletin board system. Wow, that caught me off guard.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#23

Post by notaddicted » Thu, Sep 03, 2009, 7:43 PM UTC

psadauskas wrote: post #11

He also leaves out that you could buy 10 of these Backblaze things for the price of one Sun X4540. So with a distributed load, thats 10x the performance, 10x the reliability, etc... I'm not an expert on storage, but I'd be curious to see how that stacks up, never mind the comet-proof redundancy by placing your storage in datacenters across the world.

A 48TB Sun X4540 is 50K (list price, sun.com) (2x6core, 32GB RAM, 4xGigE, 4U, 2 year warranty).

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#24

Post by mikeryan » Thu, Sep 03, 2009, 7:49 PM UTC

notaddicted wrote: post #23

psadauskas wrote: post #11

He also leaves out that you could buy 10 of these Backblaze things for the price of one Sun X4540. So with a distributed load, thats 10x the performance, 10x the reliability, etc... I'm not an expert on storage, but I'd be curious to see how that stacks up, never mind the comet-proof redundancy by placing your storage in datacenters across the world.

A 48TB Sun X4540 is 50K (list price, sun.com) (2x6core, 32GB RAM, 4xGigE, 4U, 2 year warranty).

The BackBlaze device is $8K for 67TB.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#25

Post by jacquesm » Thu, Sep 03, 2009, 7:53 PM UTC

notaddicted wrote: post #21

jacquesm wrote: post #5

Earlier quoted context omitted.

> Given the reliability info he cites, the fact that backblaze is still in business must mean their usage-to-storage ratio is small. No, it probably means that they get their reliability at a different level, such as simply storing the data on a number of machines. Systems like glusterfs make this completely transparant.

( ahhh I deleted my post before I saw the response ) From the numbers he cited I got the impression that the drives they are using would fail 3+ times more often than enterprise drives (lower MTTF + speced at low temps). The cost of replacing the drives (even if they are 1/2 the price) would make the system uneconomical. Also: because of their case design (non-hot-swappable) the cost of human replacement is higher to…

I think their use case is sufficiently a-typical to get away with this.

They're primarily write-only, only when a customer retrieves their data does stuff get read.

If they spin down the drives when the volume is not being accessed the mttf for a single drive goes up a lot, possibly beyond the point where it matters.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#26

Post by jacquesm » Thu, Sep 03, 2009, 7:54 PM UTC

sp332 wrote: post #9

jacquesm wrote: post #2

That's a pretty weird analysis, comparing a system that is special purpose with a general purpose one. He's concentrating on disk-to-board throughput, but this being network attached storage on a LAN, using the motherboards' single ethernet port the amount of data read or written to those drives per second will never exceed 1 Gbit/second anyway. I agree with the reliability analysis though, backblaze seems to be on f…

"That's a pretty weird analysis, comparing a system that is special purpose with a general purpose one." I think that's why he says right at the beginning: This device is that cheap because it cuts several corners. That's okay for them. But for general purpose this creates problems. I want to share my concerns just to show you, that you can't compare this to a X4540 device.

And then he goes and does the comparison anyway...

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#27

Post by gaius » Thu, Sep 03, 2009, 7:55 PM UTC

sophacles wrote: post #7

Ok I must be missing something. Everyone is knocking this backblaze thing because it doesnt do ZFS or because it is not a "super duper high end san replacement" or because the components are not rated for enterprise level work. It seems to me that a company whose product is personal and small business backups does not need any of those things. They have software to do mirroring. Why would they need any fast network f…

If "doing backups" is your business, then "look how cheap our kit is!" probably isn't the best marketing strategy. A "small business" is someone's bread-and-butter, they don't care that their backup needs aren't sexy to storage geeks.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#28

Post by mikeryan » Thu, Sep 03, 2009, 7:57 PM UTC

yangyang wrote: post #16

psadauskas wrote: post #11

He also leaves out that you could buy 10 of these Backblaze things for the price of one Sun X4540. So with a distributed load, thats 10x the performance, 10x the reliability, etc... I'm not an expert on storage, but I'd be curious to see how that stacks up, never mind the comet-proof redundancy by placing your storage in datacenters across the world.

10x the rack space, 10x the cooling, 10x the power ...

Its not though. Both devices are 4U rack space servers. This actually sticks 67TB in one 4U rack the X4540 gets 48TB in 4U.

This is less space rackpace per TB maybe more power.

Even if you need two of these for redundancy the cost savings is significant.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#29

Post by gaius » Thu, Sep 03, 2009, 7:59 PM UTC

modoc wrote: post #17

yangyang wrote: post #16

Earlier quoted context omitted.

10x the rack space, 10x the cooling, 10x the power ...

Those costs are minor compared to the hardware solutions being discussed.

Haha! Unexpectedly needing a new cage at your DC, or an extra 50kW power, or being told you can't put anything more in your half-full racks because you've exceeded floor loading gets expensive real quick.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#30

Post by jacquesm » Thu, Sep 03, 2009, 7:59 PM UTC

sh1mmer wrote: post #12

jacquesm wrote: post #2

That's a pretty weird analysis, comparing a system that is special purpose with a general purpose one. He's concentrating on disk-to-board throughput, but this being network attached storage on a LAN, using the motherboards' single ethernet port the amount of data read or written to those drives per second will never exceed 1 Gbit/second anyway. I agree with the reliability analysis though, backblaze seems to be on f…

I think the can of worms is that Backblaze themselves compared their system to more general solutions. While they were doing that for their own purpose they weren't so explicit with that point that many people (such as GigaOM, etc) took it as a general comparison.

They explicitly said that it was a storage unit/$ comparison, not throughput, not reliability.