Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sun engineer responds to the Backblaze "Petabytes on a budget" design

c0t0d0s0.org

Post reply on HN
Print view View original

81–90 of 109 posts

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#81

Post by Andys »

jacquesm wrote: post #30
sh1mmer wrote: post #12

Earlier quoted context omitted.

I think the can of worms is that Backblaze themselves compared their system to more general solutions. While they were doing that for their own purpose they weren't so explicit with that point that many people (such as GigaOM, etc) took it as a general comparison.

They explicitly said that it was a storage unit/$ comparison, not throughput, not reliability.

It would have helped if they'd published graphs next to it showing the same list of systems sorted by throughput and another sorted by reliability!

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#82

Post by rbanffy »

russss wrote: post #75
rbanffy wrote: post #67

Earlier quoted context omitted.

"how did so many people get the idea that this article about a specific solution to a specific problem was actually some sort of general purpose solution attacking all the big name people?" Backblaze did it, They did it when they decided to put this graph in the article: http://blog.backblaze.com/wp-content/uploads/2009/08/cost-of... It's an apples to oranges comparison.

Well no, it's not an apples to oranges comparison because no hardware vendors offer a solution which is comparable to the one Backblaze have devised. That graph is simply comparing their solution to the closest commercial equivalents. It just so happens that these are all way off because all hardware manufacturers want to design their hardware to provide the highest throughput. (That said, our experience with the Sun…

"That said, our experience with the Sun X4500s hasn't been great from an I/O point of view."

Of course it's not. It's an IBM PC down inside. I bet it can still boot MS-DOS and run GW-BASIC. ;-)

PCs are not a nice architecture for servers: there are starvation points all over the system, from registers (AMD64 solves some of it) to memory to I/O. Sun could have based the Thumper on a more server-ish (SPARC?) design, with plenty of memory and I/O bandwidth, but then it would not run Windows and the ability to run Windows is a defining advantage in the high-volume server market.

If I were to design such a box from scratch, I would couple the disks close to the network interfaces over a dedicated bus so the CPU could just say something like "hey, disk 3, drop blocks 10239 through 10300 on buffer 12 of your network controller while I go on with header parts and prepare to send it off". While I am at it I would skip the PSUs and go with DC power and a small battery, Google-style.

I fear I probably described something Thumper-ish and I will be ridiculed by someone with lots of server design experience, but that's the life of a hardware engineer who went the software very early. And I welcome such criticism ;-)

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#83

Post by docmach »

thegoleffect wrote: post #72

Seems like he's just making it easier for Backblaze to match feature to feature by listing out the differences O_o.

I'm pretty sure Sun isn't worried about Backblaze as competition in the enterprise storage market. Building cheap storage boxes is much different than building what Sun provides.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#85

Post by jacquesm »

Andys wrote: post #81
jacquesm wrote: post #30

Earlier quoted context omitted.

They explicitly said that it was a storage unit/$ comparison, not throughput, not reliability.

It would have helped if they'd published graphs next to it showing the same list of systems sorted by throughput and another sorted by reliability!

For sure. I think one of the big problems with the original article is that the fabriacator of the project did the comparison and failed to see the suggestion that was being made. Probably not on purpose, but plenty of folks ran with it thinking they could have all of 'cheap, fast, reliable'.

They just got 'cheap', reliable comes at the expense of using multiple of these units and fast looks like it is going to be impossible without striping across multiple setups like this.

Interesting for an extremely narrow use case (long tail storage, think backup services, video sites, photo sites).

Outside of that maybe useable with some adaptation but not 'as is', besides that it is not as though their machines are up for sale, the figures they quote are mostly after all the r&d is already factored out, if you had to do this for a one-off, just cutting and folding that case would cost a pretty penny.

I'm fairly sure there is a market for a device that is a more robust version of this.

Probably at a 50% or so premium compared to the one they showed you could get a wider spectrum of uses and a bit more robust construction.

I would have bought one three months ago, now I had to settle for 20T or so in one 4U box, for a price a little over half of what they quoted in the article.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#86

Post by nkurz »

rbanffy wrote: post #82
russss wrote: post #75

Earlier quoted context omitted.

Well no, it's not an apples to oranges comparison because no hardware vendors offer a solution which is comparable to the one Backblaze have devised. That graph is simply comparing their solution to the closest commercial equivalents. It just so happens that these are all way off because all hardware manufacturers want to design their hardware to provide the highest throughput. (That said, our experience with the Sun…

"That said, our experience with the Sun X4500s hasn't been great from an I/O point of view." Of course it's not. It's an IBM PC down inside. I bet it can still boot MS-DOS and run GW-BASIC. ;-) PCs are not a nice architecture for servers: there are starvation points all over the system, from registers (AMD64 solves some of it) to memory to I/O. Sun could have based the Thumper on a more server-ish (SPARC?) design, wi…

  PCs are not a nice architecture for servers: there are
  starvation points all over the system, from registers (AMD64
  solves some of it) to memory to I/O.
Could you clarify the ways in which you think a PC architecture will hobble a data server? It looks like the X4500 runs dual Opterons, which relatively speaking have no shortage of registers. And it supports 16 GB of memory, which doesn't seem too shabby. And while I'm not sure how the SATA controllers are connected to the bus, I'd be a little surprised if this was the bottleneck. Which part of the 'PC architecture' do you think is the limiting factor?

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#87

Post by sho »

sophacles wrote: post #39
gaius wrote: post #27

Earlier quoted context omitted.

If "doing backups" is your business, then "look how cheap our kit is!" probably isn't the best marketing strategy. A "small business" is someone's bread-and-butter, they don't care that their backup needs aren't sexy to storage geeks.

I dunno, it made me consider them. Instead of "we spend $2m on storage, and someday to hope to make enough money from people like you to not go out of business", its "look we aren't charging you much, and we are promising you 50 yrs of storage, this is how we can actually do that and stay in business".

I agree. Do you remember a company called Bandwagon? http://ridethebandwagon.com/

They had a big splash a couple of years ago about how they were going to offer unlimited backup for all your mp3s in iTunes, all nicely and automatically synced, for about $10/month or something. They were going to do the storage on S3. They offered anyone who mentioned them on their blog a free years' service.

http://blog.ridethebandwagon.com/post/37269480/free-accounts...

I had about 400G of mp3s at the time I think. I thought, "OK!"

Their service lasted about 2 days as they underestimated, seemingly by about an order of magnitude, how much data people actually had.

http://delong.typepad.com/sdj/2007/02/adverse_selecti.html

Then they went back to the drawing board and came up with some lame service where you buy the storage space on S3 yourself and then they sell you the ability to access it or something. Lucky they didn't go out of business.

Anyway, knowing the details of the kind of system Backblaze is using, and the price per gig it breaks down to for them, is evidence they're not overselling themselves, and gives me confidence that they'll still actually be there in a few years, unlike Bandwagon. I'm much more likely to use them because of that. Add in all the discussion they've generated, and it's been a very clever marketing event for them. Kudos!

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#89

Post by ntoshev »

dlsspy wrote: post #70
onedognight wrote: post #33

Earlier quoted context omitted.

Backblaze is making a backup server. They don't need to handle power supply failures. If one goes down, you'll replace it long before someone needs their backup. They don't need to worry about disks failing while writing (zfs and battery backup). Hell they can restart from scratch any one backup that encounters disk failures. They have all the time they want to do a backup.

> you'll replace it long before someone needs their backup How do you know when I'll need my backup?

Statistically it would be a rare case for a customer to request a backup restore from a failed unit even if you don't replicate. Bb could handle these with polite mails explaining your data will be available in 24 hours and offering you some free service as compensation - it wouldn't hurt them, they do consumer backups.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#90

Post by sho »

anigbrowl wrote: post #19
jacquesm wrote: post #2

That's a pretty weird analysis, comparing a system that is special purpose with a general purpose one. He's concentrating on disk-to-board throughput, but this being network attached storage on a LAN, using the motherboards' single ethernet port the amount of data read or written to those drives per second will never exceed 1 Gbit/second anyway. I agree with the reliability analysis though, backblaze seems to be on f…

I think his 'DC-3' analogy is valid - BB has a decent system if you don't mind the risks, Sun is selling peace of mind.

Ask Joyent about the "peace of mind" they got from their Thumper ...
Post reply on HN