Live data from Hacker News

Sun engineer responds to the Backblaze "Petabytes on a budget" design

c0t0d0s0.org

41–50 of 109 posts

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#41

He also leaves out that you could buy 10 of these Backblaze things for the price of one Sun X4540. So with a distributed load, thats 10x the performance, 10x the reliability, etc... I'm not an expert on storage, but I'd be curious to see how that stacks up, never mind the comet-proof redundancy by placing your storage in datacenters across the world.

10x the rack space, 10x the cooling, 10x the power ...

10x the capacity though - what's your point?

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#42
post #34

Earlier quoted context omitted.

The BackBlaze device is $8K for 67TB.

With 1 processor and 4G RAM.

So if you normalize this a bit, for around 50K you can get 6 BackBlazes (6 processors, 24GB of RAM and 402TB of storage).

Obviously we're still considering apples and oranges to taking this too much further would likely be silly. But its still pretty impressive numbers.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#43
post #29
post #17

Earlier quoted context omitted.

Those costs are minor compared to the hardware solutions being discussed.

Haha! Unexpectedly needing a new cage at your DC, or an extra 50kW power, or being told you can't put anything more in your half-full racks because you've exceeded floor loading gets expensive real quick.

At svtix, a tier3 data center in downtown san jose, Not at the top end of the price scale, but not at the bottom, I can get 2 20a 120v circuits (4800w, that's 3360 usable.) and a full locking rack for around $1100 a month. I can buy two or three racks full of power. If I wanted to pull a gig connection to another nearby data center, say, because I couldn't get more at svtix, we are talking something like $500-$1000 a month, depending on how badly I get screwed by my poor negotiation skills.

Just sayin, at those prices, it takes a long time to amortize out a six figure appliance.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#44
post #2

That's a pretty weird analysis, comparing a system that is special purpose with a general purpose one. He's concentrating on disk-to-board throughput, but this being network attached storage on a LAN, using the motherboards' single ethernet port the amount of data read or written to those drives per second will never exceed 1 Gbit/second anyway. I agree with the reliability analysis though, backblaze seems to be on f…

Can anyone estimate the practical benefits (or not) of adding some extra Gbit network cards in the free motherboard slots on the BB device?

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#45
post #29
post #17

Earlier quoted context omitted.

Those costs are minor compared to the hardware solutions being discussed.

Haha! Unexpectedly needing a new cage at your DC, or an extra 50kW power, or being told you can't put anything more in your half-full racks because you've exceeded floor loading gets expensive real quick.

Upvoted. I would upvote a million times over if I could. So many developers seem to not realize just how expensive physical plant (and the people to administer it) really is -- "hardware is cheap" fails to account for the facility costs involved in acquiring necessary space, power, cooling to run it -- often ahead of time.

Managing the IT team and a growing data center installation for a medium size business was one of the most enlightening experiences I've had as a developer writing software targeted at "internet-scale".

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#46
post #43
post #29

Earlier quoted context omitted.

Haha! Unexpectedly needing a new cage at your DC, or an extra 50kW power, or being told you can't put anything more in your half-full racks because you've exceeded floor loading gets expensive real quick.

At svtix, a tier3 data center in downtown san jose, Not at the top end of the price scale, but not at the bottom, I can get 2 20a 120v circuits (4800w, that's 3360 usable.) and a full locking rack for around $1100 a month. I can buy two or three racks full of power. If I wanted to pull a gig connection to another nearby data center, say, because I couldn't get more at svtix, we are talking something like $500-$1000 a…

At svtix, a tier3 data center in downtown san jose, Not at the top end of the price scale, but not at the bottom, I can get 2 20a 120v circuits and a full locking rack for around $1100 a month. I can buy two or three racks full of power.

As long as that space is available. To guarantee business capacity you'll probably need to buy a large block of rackspace at one time. Of course, capacity availability varies based on the current local business climate, but that's something you have to plan for too.

If I wanted to pull a gig connection to another nearby data center, say, because I couldn't get more at svtix, we are talking something like $500-$1000 a month

Upwards of $1000, but note that GigE is a fairly small link for data center cross-connection and will still usually have a large lead-time. Not only do you need to be able to guarantee sufficient capacity either at svtix or at the other colo, but you need to be prepared to wait to get that capacity allocated for your use.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#48
post #43

Earlier quoted context omitted.

At svtix, a tier3 data center in downtown san jose, Not at the top end of the price scale, but not at the bottom, I can get 2 20a 120v circuits (4800w, that's 3360 usable.) and a full locking rack for around $1100 a month. I can buy two or three racks full of power. If I wanted to pull a gig connection to another nearby data center, say, because I couldn't get more at svtix, we are talking something like $500-$1000 a…

At svtix, a tier3 data center in downtown san jose, Not at the top end of the price scale, but not at the bottom, I can get 2 20a 120v circuits and a full locking rack for around $1100 a month. I can buy two or three racks full of power. As long as that space is available. To guarantee business capacity you'll probably need to buy a large block of rackspace at one time. Of course, capacity availability varies based o…

you have a very good point that you pay much higher prices (or just massively overbuy, which is similar) if you want guaranteed capacity right now, right were you are.

My model is such that it's not that big of a deal to just get an unconnected rack at a new data center and buy transit there. When I was blocked on new power at SVTIX, I slapped a few new servers in at rippleweb.com, which is in Sacramento. If I had a big, expensive san I couldn't replicate, well, then I'd have a different model. (rippleweb has awesome prices on high-power 1U server co-location, but it's not as good as getting a full rack elsewhere.)

But yes, the observation that you pay more if you want your stuff provisioned right now is very correct. At prgmr.com, well, I wait for the deals, so I end up paying a lot less than I would otherwise. (but then, sometimes the 'we are out of space, come back later' message is up for longer than I'd like, as well. It's definitely a compromise.)

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#49
post #2

That's a pretty weird analysis, comparing a system that is special purpose with a general purpose one. He's concentrating on disk-to-board throughput, but this being network attached storage on a LAN, using the motherboards' single ethernet port the amount of data read or written to those drives per second will never exceed 1 Gbit/second anyway. I agree with the reliability analysis though, backblaze seems to be on f…

I think his 'DC-3' analogy is valid - BB has a decent system if you don't mind the risks, Sun is selling peace of mind.

FUD and RAS being two of the basic tenets of marketing most anything in the enterprise IT space, after all.

Re: Sun engineer responds to the Backblaze "Petabytes on a budget" design

#50

Earlier quoted context omitted.

10x the rack space, 10x the cooling, 10x the power ...

People often forget that the rack space, the actual physical space, is very costly. Depending on the datacenter, the power might run you just as much too. Last company I worked at was essentially forced out for using too much power per U.

To give people an idea; our main datacenter gives us 2kW per rack, and we pay extra for 4kW. A rack full of these ~700W storage pods would need about double that again, and isn't going to leave you much wiggle room.
Post reply on HN