Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to: Compete with Amazon S3 without Buying Hardware

blog.adamsmith.cc

Post reply on HN
Print view View original

21–30 of 40 posts

Re: How to: Compete with Amazon S3 without Buying Hardware

#21

Post by lusr »

driverdan wrote: post #17

I've been watching the storage industry for years as a hobby-passion. Adam pretty much hit all the major points. The storage space is still open for disruption but is hard with high risk. It's not like building a website. You need serious funding for hardware. You need people to manage the hardware. You need complex software to manage the data and ensure security. One serious breach early on and you're done. Competin…

I imagine that the response times of S3 vs. this Frankenstore model will be much better, which may be an issue for certain applications.

Also you need to host infrastructure software that knows where your data is sitting, how to deal with provider failures, how to efficiently route requests, etc. which means yet-another-thing-to-configure.

Finally, if the volume of data you're storing is so expensive on S3, I have to wonder why you have all this non-revenue generating data stored in the first place. Processing it also seems more expensive now because the free bandwidth you get from EC2S3 won't apply in the Frankenstore model.

Re: How to: Compete with Amazon S3 without Buying Hardware

#22

Post by wmf »

notmyname wrote: post #20
adamsmith wrote: post #6

Earlier quoted context omitted.

> This article is basically advocating RAID 5 across many storage providers. That is correct. And to be more precise, I'm advocating RAID 5 across storage providers as a service , so people who just want to store data don't have to manage anything.

If this were built on top of (or in conjunction with) Openstack swift ( http://swift.openstack.org ), it could be done with a common storage backend. There are already several public cloud storage providers using swift (Rackspace, HP, Softlayer, Internap, KT, and others), and a growing number of private deployments as well (Wikipedia probably being the most recognizable name). If you'd like to talk more about this, s…

Aren't most Swift providers already using 2x-3x duplication? Putting parity on top of that would be like RAID15 (the reason you've never heard of RAID15 is that it would be crazy expensive).

Re: How to: Compete with Amazon S3 without Buying Hardware

#23

Post by notmyname »

wmf wrote: post #22
notmyname wrote: post #20

Earlier quoted context omitted.

If this were built on top of (or in conjunction with) Openstack swift ( http://swift.openstack.org ), it could be done with a common storage backend. There are already several public cloud storage providers using swift (Rackspace, HP, Softlayer, Internap, KT, and others), and a growing number of private deployments as well (Wikipedia probably being the most recognizable name). If you'd like to talk more about this, s…

Aren't most Swift providers already using 2x-3x duplication? Putting parity on top of that would be like RAID15 (the reason you've never heard of RAID15 is that it would be crazy expensive).

Yes, swift provides multiple redundant copies of the data. It depends on what you are protecting against: failure of a piece of your data or failure of a hosting provider.

If you simply rely on "dumb disks" spread across multiple providers to provide availability, then you may be interested in looking at the nova-volumes part of openstack (it provides block storage attached to nova VMs). As part of openstack, it's an open system that is seeing rapid adoption.

However, one of the most-requested features for swift is to provide support for logical clusters that span a wide geographic area. This could potentially allow multiple providers to collaborate in providing a multi-provider storage system. However, I'd guess that the technical problems are much simpler than the business problems in setting up multi-provider clusters.

Re: How to: Compete with Amazon S3 without Buying Hardware

#24

Post by adamsmith »

lusr wrote: post #21
driverdan wrote: post #17

I've been watching the storage industry for years as a hobby-passion. Adam pretty much hit all the major points. The storage space is still open for disruption but is hard with high risk. It's not like building a website. You need serious funding for hardware. You need people to manage the hardware. You need complex software to manage the data and ensure security. One serious breach early on and you're done. Competin…

I imagine that the response times of S3 vs. this Frankenstore model will be much better, which may be an issue for certain applications. Also you need to host infrastructure software that knows where your data is sitting, how to deal with provider failures, how to efficiently route requests, etc. which means yet-another-thing-to-configure. Finally, if the volume of data you're storing is so expensive on S3, I have to…

All good points!

I should have mentioned this more explicitly but you could take the buying-raw-storage model and use it to do anything S3 does, I think. Eg you could have three independent whole copies, or one whole copy and 1.5 copies distributed widely.

The only thing I can think of that Amazon could do that you couldn't do, if the raw storage providers are untrusted, is serve the data with no addition hops, since the data would be encrypted.

Re: How to: Compete with Amazon S3 without Buying Hardware

#26

Post by coalsoul »

Break the file into 3 equal parts. You only need to store one part plus the parity file. And not necessarily with the same cloud provider. Result: you pay 1/3 less for storage. So if Dropbox is so clever why aren't they doing this for their customers? xor'ing/Reed-Solomon has been used this way since Usenet.

Re: How to: Compete with Amazon S3 without Buying Hardware

#27

Post by thereallurch »

wmf wrote: post #18
thereallurch wrote: post #13

Earlier quoted context omitted.

provider1=[d1,1][d1,2][d1,3] provider2=[d2,1][d2,2][d2,3] provider3=[d3,1][d3,2][d3,3] Each drive cost $x. You have 9$x = cost to store 1 drive of data, across 3 providers, who each store 3 copies. If drive prices halve, it's still 9$x.

I don't think anyone is proposing 3 providers, who each store 3 copies . Either use one provider who stores 3 copies (Amazon) or three providers who each store one copy.

"People always conveniently forget as well that when you store data at S3 or Rackspace they're doing 3 replica's. If you store 3TB's, its not just the cost of a single 3TB drive..its 3x. To do it at home you'd need to buy 3x3TB Drives @300$ each."

Existing companies store multiple copies transparently...Either way, 1 copy or 3 copies, I dont think the math is wrong. Just change from 9$x to 3$x.

Re: How to: Compete with Amazon S3 without Buying Hardware

#28

Post by Dylan16807 »

Terretta wrote: post #10
adamsmith wrote: post #6

Earlier quoted context omitted.

> This article is basically advocating RAID 5 across many storage providers. That is correct. And to be more precise, I'm advocating RAID 5 across storage providers as a service , so people who just want to store data don't have to manage anything.

>> This article is basically advocating RAID 5 across many storage providers. > That is correct. The diagram in the article and text description is RAID 1+0 aka RAID 10.

No, it's RAID 5. Well, specifically, it's RAID 6 but customizable. You can lose any k drives.

Re: How to: Compete with Amazon S3 without Buying Hardware

#29

Post by mbell »

driverdan wrote: post #17

I've been watching the storage industry for years as a hobby-passion. Adam pretty much hit all the major points. The storage space is still open for disruption but is hard with high risk. It's not like building a website. You need serious funding for hardware. You need people to manage the hardware. You need complex software to manage the data and ensure security. One serious breach early on and you're done. Competin…

It really isn't that expensive see http://www.backblaze.com/ and specifically http://blog.backblaze.com/2011/07/20/petabytes-on-a-budget-v...

Obviously they are targeted at backups but you wouldn't need to change a lot to improve performance (mostly it would be in software + some caching boxes I think).

Re: How to: Compete with Amazon S3 without Buying Hardware

#30

Post by blantonl »

soult wrote: post #15

Tahoe-LAFS[1] is a storage system that works similarly. It splits data into n fragments, of which any k fragments are enough to restore the data. This leads to a replication factor of only DATA_SIZE/k*n, while you can still lose n-k pieces without data loss. Additionally all data is encrypted, signed and optionally deduplicated. One of the authors of Tahoe-LAFS started a company that ported the whole system over to c…

Now this is a cloud application.
Post reply on HN