You ask about online bulk storage at reasonable rates, but you've not mentioned what youre acceptable rate of data loss is?
How much data are you willing to lose per day out of your 100TB?
Ask HN: Cheap Bulk Storage?
21–30 of 43 posts
Re: Ask HN: Cheap Bulk Storage?
#22Earlier quoted context omitted.
Yeah, basically 2.5" gives you the same space but twice the spindles for twice the price. Probably not worth it for archival storage.
Agreed. It'd be faster, but that's not really what we're looking for- We want to get the cost per GB to be as low as humanly possible, while maintaining at least an illusion of reliability.
So store it all on one of those fake Chinese USB drives. That gives a nice illusion, and is incredibly cheap...
Alternatively we could consider what level of reliability you want and how that will be tested.
Re: Ask HN: Cheap Bulk Storage?
#23Earlier quoted context omitted.
Agreed. It'd be faster, but that's not really what we're looking for- We want to get the cost per GB to be as low as humanly possible, while maintaining at least an illusion of reliability.
So store it all on one of those fake Chinese USB drives. That gives a nice illusion, and is incredibly cheap... Alternatively we could consider what level of reliability you want and how that will be tested.
In my calculations I was doing elsewhere in the thread, I was assuming 1 drive out of 4 would be for redundancy. There's nothing on these drives that's irreplaceable, and we have system-level redundancy.
Re: Ask HN: Cheap Bulk Storage?
#24Diomede might be worth a look. Their offline and nearline options are waaay less expensive than S3. Never used them though so let us know how it works out if you go with them. http://www.diomedestorage.com/
Their page appears to have a single unlinked image on it, and nothing else..?
http://techcrunch.com/2009/02/27/diomede-offers-green-file-s... seems to offer some more details. Their site leaves an awful lot to be desired.
Re: Ask HN: Cheap Bulk Storage?
#25You ask about online bulk storage at reasonable rates, but you've not mentioned what youre acceptable rate of data loss is? How much data are you willing to lose per day out of your 100TB?
I'd be willing to lose 100TB at once, assuming it wasn't every day.
If there were a system where it lost even 20TB/day, I'd be interested, depending on what the tradeoffs were! I'd expect that to be pretty seriously cheap to offset it.
That's why I'm asking what options people know about. If you know one that is randomly lossy but cheap, I'd be REALLY curious as to what their backend is!
Re: Ask HN: Cheap Bulk Storage?
#26Earlier quoted context omitted.
It's also worth looking at their 72 disk, 2.5" version http://www.supermicro.com/products/chassis/4U/417/SC417E16-R... Using that, and 72 1TB 2.5" drives, ( http://www.newegg.com/Product/Product.aspx?Item=N82E16822136... ) (($109 * 72) / (72 * 1TB * .75) in dollars per gigabyte) = $.15/Gb
It's worth noting at this time, that most SSDs only come in a 2.5" version. (Not for cheap bulk storage but because that's arguable Supermicro's intentions for this chassis.)
No its not intended for SSDs. There is not nearly enough IO bandwidth for SSD to make any sense. 2.5 inch hard drives are what you use if you want disks with spindles not just big storage. Eg for virtualised database servers.
Re: Ask HN: Cheap Bulk Storage?
#27Earlier quoted context omitted.
Their page appears to have a single unlinked image on it, and nothing else..?
http://techcrunch.com/2009/02/27/diomede-offers-green-file-s... seems to offer some more details. Their site leaves an awful lot to be desired.
Ah, yes. I remember reading about them now. I'd be interested in looking over their offerings, if they ever release any.
Re: Ask HN: Cheap Bulk Storage?
#28Would be interesting to install OpenStack Swift on top of Backblaze storage and see how it works.
I only looked at it briefly, but isn't Swift for multi-system installs, not multi-spindle?
Re: Ask HN: Cheap Bulk Storage?
#29SpiderOak (the backup and sync company) is on the verge of launching a S3-like service, tuned for for archival class data. It's in beta now.
It's also entirely open source, so you could run it on your own hardware if you wanted. It uses parity instead of replication for storing data (but with arbitrary effectively replication levels.) So with R=3, you need a minimum of about 10 nodes to be efficient.
The beta uses AMQP internally, and the upcoming release version is Python/zeromq/gevent based.
Re: Ask HN: Cheap Bulk Storage?
#30I'm surprised there aren't any services selling raw storage space. i.e. no RAID or other redundancy, no doubling as a static web-server, just a bunch of disks. So people (myself included) buy physical HDDs for local backup -- simply because every online alternative jacks up the price by maintaining 3+ copies and keeping everything permanently online.
That's a curious idea. How would you access the raw storage? iSCSI? And then you run your own RAID on top? What's the advantage to you? It's ever so slightly cheaper? Backblaze (no affiliation) is $5/month or $50/yr. That's pretty hard to beat. Also, running a static web-server is fairly trivial to setup these days, what would a service not running a static web-server do/have? Every online alternative has a vested in…
I need data backup on a personal level rather than for business reasons, and I love what Backblaze offer... except that for my data and with my ~50kbps upload speed, it would take about five years to upload it all to their servers.