Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Let's talk about a hypothetical public-facing service”

reddit.com

Post reply on HN
Print view View original

21–30 of 72 posts

Re: “Let's talk about a hypothetical public-facing service”

#21

Post by rosser »

...Extremely Massive Corporation...

I knew EMC storage was utter shit when, upon attempting to create a new RAID group, I realized that the configuration tool's default was to stripe across drives within a shelf, not to create stripes that span shelves.

Worse, to create the more fault-tolerant, shelf-spanning RAID volumes, one must manually add drives, one by one to the array, in a process that involves about 44 (slight hyperbole) clicks per disk.

And then there was the fact that the configuration tool was Windows-only.

Yeah, screw those guys.

Re: “Let's talk about a hypothetical public-facing service”

#22

Post by mrud »

ThePhysicist wrote: post #19
smacktoward wrote: post #5

> You have a moment where you envision the future of virtualized storage and think about how great it will be when storage is nearly free and outsourceable and you can stop buying disks from Amazon every few months The future is now! Instead of you sending money to Amazon and them sending you disks, they keep the disks and you send them the money anyway. Progress! :-D

Sadly, depending on what you want to accomplish with your storage the future is still not here. For example, if you want to share a virtual disk with multiple servers through NFS you will still need to manually create a NFS high-availability cluster using e.g. DRBD on most cloud providers (e.g. EC2 or Azure). To be fair, MS Azure offers fully managed SMB shares as a service, but the transfer speed is quite low (60 MB…

AWS has EFS [1] which should allow you to share one FS to multiple serves. It is still in preview but could help address these needs.

[1] http://aws.amazon.com/efs/pricing/

Re: “Let's talk about a hypothetical public-facing service”

#23

Post by idunno246 »

ThePhysicist wrote: post #19
smacktoward wrote: post #5

> You have a moment where you envision the future of virtualized storage and think about how great it will be when storage is nearly free and outsourceable and you can stop buying disks from Amazon every few months The future is now! Instead of you sending money to Amazon and them sending you disks, they keep the disks and you send them the money anyway. Progress! :-D

Sadly, depending on what you want to accomplish with your storage the future is still not here. For example, if you want to share a virtual disk with multiple servers through NFS you will still need to manually create a NFS high-availability cluster using e.g. DRBD on most cloud providers (e.g. EC2 or Azure). To be fair, MS Azure offers fully managed SMB shares as a service, but the transfer speed is quite low (60 MB…

Ec2 has efs at least, no idea if it's any good

Re: “Let's talk about a hypothetical public-facing service”

#24

Post by [deleted] »

ThePhysicist wrote: post #19
smacktoward wrote: post #5

> You have a moment where you envision the future of virtualized storage and think about how great it will be when storage is nearly free and outsourceable and you can stop buying disks from Amazon every few months The future is now! Instead of you sending money to Amazon and them sending you disks, they keep the disks and you send them the money anyway. Progress! :-D

Sadly, depending on what you want to accomplish with your storage the future is still not here. For example, if you want to share a virtual disk with multiple servers through NFS you will still need to manually create a NFS high-availability cluster using e.g. DRBD on most cloud providers (e.g. EC2 or Azure). To be fair, MS Azure offers fully managed SMB shares as a service, but the transfer speed is quite low (60 MB…

[deleted]

Re: “Let's talk about a hypothetical public-facing service”

#25

Post by glass- »

intrasight wrote: post #3

Got a good chuckle. Such is life in IT. Gotta admit I missed the "Extremely Massive Corporation" hint.

It reminds me of Douglas Crockford's talk where he mentioned that a company asked him for an exemption to the "do no evil" clause in the JSON License, and he said he didn't want to name the company as that would embarrass them so he would instead give their initials: IBM.

Re: “Let's talk about a hypothetical public-facing service”

#26

Post by nadams »

smacktoward wrote: post #5

> You have a moment where you envision the future of virtualized storage and think about how great it will be when storage is nearly free and outsourceable and you can stop buying disks from Amazon every few months The future is now! Instead of you sending money to Amazon and them sending you disks, they keep the disks and you send them the money anyway. Progress! :-D

> you can stop buying disks from Amazon every few months

This is something that has been puzzling me. Many years ago I purchased 4x 2TB 5900 RPM drives for a 4 bay ReadyNAS (cost about ~$300 for drives plus ReadyNAS). They have been spinning nonstop for ~4 years [1] and haven't had to replace a single one. Not even an increase in errors to signal that the drive is going.

Yet - I've worked on a SAN that cost hundreds of thousands of dollars and would have to replace a disk about every month.

Granted the disks in SANs probably spin faster (thus faster data access/lower MTBF) - but that high failure rate seems rather suspicious to me.

[1] http://i.imgur.com/NRQCedj.png

Re: “Let's talk about a hypothetical public-facing service”

#27

Post by Sanddancer »

windsurfer wrote: post #18
smacktoward wrote: post #5

> You have a moment where you envision the future of virtualized storage and think about how great it will be when storage is nearly free and outsourceable and you can stop buying disks from Amazon every few months The future is now! Instead of you sending money to Amazon and them sending you disks, they keep the disks and you send them the money anyway. Progress! :-D

If the disks only last so long, is it really so terrible that Amazon gets to keep the broken outdated junk?

Yes. One can build a system that holds a petabyte of data, fits in less than one rack, has multiple redundancy, for about a hundred grand these days. Multiplied by three locations, that'll get you to $300000. Add in another $35000 for three years of hosting at three different datacenters, and you get to $335000. Sounds like a lot, right?

Amazon's even more expensive. Storing one petabyte on Amazon, at the slowest, cheapest, glacier level, will cost you $10000/month, so at about 10 months, you've paid a hundred grand. At thirty six months, you've paid three hundred sixty grand. Plus, you have no hardware to show for it, while amazon sells that broken outdated junk to a reseller to regain some of their investment. Had you hosted it yourself, you could have sold all that equipment off to refurbishers and resellers and regained at least a portion of your investment back. So yeah, it is pretty terrible to choose Amazon.

Re: “Let's talk about a hypothetical public-facing service”

#28

Post by khuey »

Sanddancer wrote: post #27
windsurfer wrote: post #18

Earlier quoted context omitted.

If the disks only last so long, is it really so terrible that Amazon gets to keep the broken outdated junk?

Yes. One can build a system that holds a petabyte of data, fits in less than one rack, has multiple redundancy, for about a hundred grand these days. Multiplied by three locations, that'll get you to $300000. Add in another $35000 for three years of hosting at three different datacenters, and you get to $335000. Sounds like a lot, right? Amazon's even more expensive. Storing one petabyte on Amazon, at the slowest, ch…

Commodity cloud services (and pricing) are meant for the 99%, not the outliers. If you have a petabyte of data to store you should either be doing enough other business with Amazon that you can negotiate rates down or you should be rolling your own.

Re: “Let's talk about a hypothetical public-facing service”

#29

Post by sneak »

Sanddancer wrote: post #27
windsurfer wrote: post #18

Earlier quoted context omitted.

If the disks only last so long, is it really so terrible that Amazon gets to keep the broken outdated junk?

Yes. One can build a system that holds a petabyte of data, fits in less than one rack, has multiple redundancy, for about a hundred grand these days. Multiplied by three locations, that'll get you to $300000. Add in another $35000 for three years of hosting at three different datacenters, and you get to $335000. Sounds like a lot, right? Amazon's even more expensive. Storing one petabyte on Amazon, at the slowest, ch…

It costs about a hundred and fifty grand annually (fully loaded, base salary 80-90k) to have the lowest end person who could conceivably keep one of these things running well for three years.

n+1 means you need two of them in case one gets the flu.

Tell me again about how Amazon's a ripoff, please.

Re: “Let's talk about a hypothetical public-facing service”

#30

Post by ChuckMcM »

nadams wrote: post #26
smacktoward wrote: post #5

> You have a moment where you envision the future of virtualized storage and think about how great it will be when storage is nearly free and outsourceable and you can stop buying disks from Amazon every few months The future is now! Instead of you sending money to Amazon and them sending you disks, they keep the disks and you send them the money anyway. Progress! :-D

> you can stop buying disks from Amazon every few months This is something that has been puzzling me. Many years ago I purchased 4x 2TB 5900 RPM drives for a 4 bay ReadyNAS (cost about ~$300 for drives plus ReadyNAS). They have been spinning nonstop for ~4 years [1] and haven't had to replace a single one. Not even an increase in errors to signal that the drive is going. Yet - I've worked on a SAN that cost hundreds…

AFR buddy, AFR. AFR is the annual failure rate, its typically 2 - 5% of the population per year. 4 drives you don't have a large enough set to see this in action, just every day you're in danger of losing a drive by a small statistical amount.

In the Blekko cluster we have just under 10,000 drives. We have a two 20 drive 'boxes' (40 drives) from Western Digital, as drives fail we pull replacements from the 'new/refurbished' box, and we put the dead one in the outgoing box. When we get up to 20 we RMA them in bulk, 20 go out, 20 more come in. That becomes the new 'new/refurbished' box.

It really isn't SAN vs non-SAN it is all statistics.

That said, if you're running your ReadyNAS with raid 10 (mirrored drives in a RAID 0 config) you may find some unpleasantness when a drive does fail. Statistically you have a 1/10 chance of not being able to re-silver the mirror for a 5900 RPM desktop SATA drive. That gets a bit painful.

Post reply on HN