Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sun engineer responds to the Backblaze "Petabytes on a budget" design

c0t0d0s0.org

Post reply on HN
Print view View original

51–60 of 109 posts

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#51

Post by lsc »

anigbrowl wrote: post #44
jacquesm wrote: post #2

That's a pretty weird analysis, comparing a system that is special purpose with a general purpose one. He's concentrating on disk-to-board throughput, but this being network attached storage on a LAN, using the motherboards' single ethernet port the amount of data read or written to those drives per second will never exceed 1 Gbit/second anyway. I agree with the reliability analysis though, backblaze seems to be on f…

Can anyone estimate the practical benefits (or not) of adding some extra Gbit network cards in the free motherboard slots on the BB device?

You would have to do work (either 'etherchannel' or similar bonding, or load balance requests over 2 IPs) but in theory, if you had a high-bandwidth bus like pci-express, you should be able to get more bandwidth out of a chassis by adding more ethernet cards.

Me, I would add more cache at the same time. SATA tends to be really good for sequential, but be really, really bad for random. multiple sequential streams hitting the same disk start to look an awful lot like random access after a while.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#52

Post by Retric »

akamaka wrote: post #40
jacquesm wrote: post #26

Earlier quoted context omitted.

And then he goes and does the comparison anyway...

Let's not forget that BackBlaze started this, by making a graph that compared their prices to S3, etc. (and were also criticised for that in the previous HN thread).

They are talking about cloud storage. Having 3 live copy's of your data in 3 locations is far more important than having lot's of redundancy at a single location. If you want redundancy just add more locations. Large scale company's like Google, Microsoft, and Yahoo understand this but sun is selling feel good hardware to companies that like to think they have unusual needs.

As to speed, nothing you do is going to reduce a disk's low latency so talking about bandwidth is mostly a waste of time. Device bandwidth is rarely the limitation once you start to scale, because you can add devices to your network faster than you can build a better network.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#53

Post by lsc »

abyssknight wrote: post #18
yangyang wrote: post #16

Earlier quoted context omitted.

10x the rack space, 10x the cooling, 10x the power ...

People often forget that the rack space, the actual physical space, is very costly. Depending on the datacenter, the power might run you just as much too. Last company I worked at was essentially forced out for using too much power per U.

yeah, cooling systems are designed to handle a certain amount of heat per square foot. Most of the he.net locations around here can only handle 1 15a 120v circut per rack. That's 5 low-power dual-socket servers for a full-sized rack.

When I'm buying data center space I look at cost per watt and ignore the space requirements. I've not yet hit a place that was willing to sell me more power than I could use in a rack. If the space is low/medium density, I fill it with 2 or 3u chassis (I'm currently using http://supermicro.com/products/chassis/3U/833/SC833S2-550.cf... because I got a good deal on a lot of them.) If it looks like they will sell me more power than I can eat with those, I pop in a few of the supermicro 2 in 1u boxes: http://www.supermicro.com/Aplus/system/1U/1021/AS-1021TM-T+....

This is part of why you see so many people leaving airspace between 1u servers. (me, I use the 3u servers for 'air gaps' as they run really cool.)

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#54

Post by wmf »

ShabbyDoo wrote: post #37

I used to work for a large consumer photo site, and we wished we could trade speed for lower cost or lower power consumption. Forget SCSI drives, SATA disks were 10x faster than what we needed for long-tail storage. If a photo became popular, we could cache it forward. The parato-ization of the photos was probably 99.9/0.1. I'm actually surprised that drive manufacturers haven't come out with a slower disk that is ch…

Seagate Barracuda LP and WD RE-GP are ~6000 RPM disks that are lower power than normal 7200 RPM SATA. Maybe next year they'll come out with ~5000 RPM disks.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#55

Post by lsc »

barryrandall wrote: post #36

At first don't compare the mentioned list prices with the street prices for components. This is why small companies, startups, and people new to the storage world hate working with storage vendors. I understand the economics, I get why they sell storage the way they do, and I like my 70% discount off list price, but it makes for a horrible user experience. If I were a cleverer person, I'd apply an every-day low price…

I went with the reseller I did at svtix, primarily because they were upfront enough to list prices on their website. http://egihosting.com - I don't want to spend time fucking around over price, and I'm sure you don't want me to waste your time. Post the price, and I'll buy it or I won't.

Seriously, If you say you want six figures for your product, I'm walking. I don't have that kind of scratch, and I'm not going to waste your time or my time lowballing you. If you really mean $30K, well, I'd have to scrimp and save for a few months, but it's doable if the product really does solve more than $30K worth of problems for me.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#56

Post by rbanffy »

Hoff wrote: post #49
anigbrowl wrote: post #19

Earlier quoted context omitted.

I think his 'DC-3' analogy is valid - BB has a decent system if you don't mind the risks, Sun is selling peace of mind.

FUD and RAS being two of the basic tenets of marketing most anything in the enterprise IT space, after all.

There is a lot of good engineering in the Thumper. There is a lot of clever engineering in the Backblaze. The comparison states a lot of valid points on power distribution, drive reliability and RAID throughput.

I see Backblaze will learn a lot from building their storage systems and future versions may be much better, but this one cuts too many corners for me to feel confident it will have the same kind of reliability I would recommend.

What amazes me most is the use of desktop-grade hard drives. Not because they are sloppily built, but because their performance requirements are so different from a server environment.

This makes for some nice reading: http://cacm.acm.org/magazines/2009/6/28493-hard-disk-drives-...

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#58

Post by jacquesm »

anigbrowl wrote: post #44
jacquesm wrote: post #2

That's a pretty weird analysis, comparing a system that is special purpose with a general purpose one. He's concentrating on disk-to-board throughput, but this being network attached storage on a LAN, using the motherboards' single ethernet port the amount of data read or written to those drives per second will never exceed 1 Gbit/second anyway. I agree with the reliability analysis though, backblaze seems to be on f…

Can anyone estimate the practical benefits (or not) of adding some extra Gbit network cards in the free motherboard slots on the BB device?

They are using a consumer grade motherboard with one ethernet port, 3 PCI slots and one PCI express slot, the slots are all used up by the sata interfaces.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#59

Post by jacquesm »

antonovka wrote: post #46
lsc wrote: post #43

Earlier quoted context omitted.

At svtix, a tier3 data center in downtown san jose, Not at the top end of the price scale, but not at the bottom, I can get 2 20a 120v circuits (4800w, that's 3360 usable.) and a full locking rack for around $1100 a month. I can buy two or three racks full of power. If I wanted to pull a gig connection to another nearby data center, say, because I couldn't get more at svtix, we are talking something like $500-$1000 a…

At svtix, a tier3 data center in downtown san jose, Not at the top end of the price scale, but not at the bottom, I can get 2 20a 120v circuits and a full locking rack for around $1100 a month. I can buy two or three racks full of power. As long as that space is available. To guarantee business capacity you'll probably need to buy a large block of rackspace at one time. Of course, capacity availability varies based o…

The use case for a company like backblaze though allows you to buy bandwidth very cheap because it will be mostly upstream, the direction in which datacenters are normally not seeing much traffic.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#60

Post by uibuibk »

anigbrowl wrote: post #19
jacquesm wrote: post #2

That's a pretty weird analysis, comparing a system that is special purpose with a general purpose one. He's concentrating on disk-to-board throughput, but this being network attached storage on a LAN, using the motherboards' single ethernet port the amount of data read or written to those drives per second will never exceed 1 Gbit/second anyway. I agree with the reliability analysis though, backblaze seems to be on f…

I think his 'DC-3' analogy is valid - BB has a decent system if you don't mind the risks, Sun is selling peace of mind.

>Sun is selling peace of mind

Right upto the point where they are sold to a database company that isn't interested in selling disks.

the nice thing about the blaze is that if seagate decide to double their prices or much you to a new expensive platform - you just buy toshiba instead.

Post reply on HN