Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Glacier

aws.amazon.com

Post reply on HN
Print view View original

321–330 of 393 posts

Re: Amazon Glacier

#321

Post by kfury »

ricardobeat wrote: post #318
tomkarlo wrote: post #309

Earlier quoted context omitted.

Having a multi-hour delay in retrieval lets them move it into their off-peak hours. Since their bandwidth costs are probably calculated off their peak usage, a service that operates entirely in the shadow of that peak has little to no incremental cost to them.

If every request is delayed by the same amount, then it just moves the peak. Otherwise it just flattens/spreads it around peak_length + 4 .

I think the point is that they're trying to average the load across the datacenter, only part of which is Glacier. If they can offset all the Glacier requests by 12 hours, they'll help normalize load when combined with non-Glacier activity.

Re: Amazon Glacier

#322

Post by Auguste »

jwr wrote: post #80
ghshephard wrote: post #36

I'm a long time user of backblaze, and I'm a big fan of the product - it does a great job of always making sure my working documents are backed up, particularly when I'm traveling overseas, and my laptop is more vulnerable to theft or damage. With that said - Backblaze is optimized for working documents - and the default "exclusion" list makes it clear they don't want to be backing up your "wab~,vmc,vhd,vo1,vo2,vsv,v…

What's even more important, you will be able to encrypt your backups without having to disclose the encryption key in case you ever need to restore (client-side encryption and decryption). This is not the case with Backblaze, which is why I switched to CrashPlan — but I'm still looking for other solutions.

Backblaze does support the optional use of a separate encryption key which they claim to never store, but there's no information about whether it is ever disclosed to them or not. There's little information on their website, but it can be set in Settings > Security in the Backblaze client.

Re: Amazon Glacier

#323

Post by Dysanovic »

nemesisj wrote: post #8
terhechte wrote: post #6

This is fantastic. I've long searched for a solution like that. This is really suitable for a remote backup that only needs to be accessed if something really bad happens (i.e. a fire breaking out, etc). I'm a lone entrepreneur, so I do have backup hard disks here, but being able to additionally save this data in the cloud is great. I'm often creating pretty big media assets, so Dropbox doesn't necessarily offer enou…

Agreed - I'm really happy about this. I have a home NAS solution that is a few TB and it's too expensive to store on S3. This is perfect to prevent the "house burned down" scenario on very large storage devices!

I'm trying to work out the best way to back-up my NAS to the cloud just in case my house burns down. It's really annoying how lots of cloud storage solutions dont cater for home users with a NAS. I agree that this could be a good, cheap way of backing stuff up.

Re: Amazon Glacier

#324

Post by tedunangst »

DeepDuh wrote: post #319
g-garron wrote: post #206

Earlier quoted context omitted.

Yes I do it, I have the weekly and monthly. And yes, I might have daily-monday / daily/tuesday.... and so on, and it would work better, but this way works for me. There is a lot of room for improvement, and it is not hard to implement with cron-jobs and more buckets.

Do you know about worst case analysis? Let's say your data loss occurs on the Sunday, 31th of a month at 23:55 and then gets synched across all your S3 backups (or it could occur at some point before that but you don't notice it). And poof goes your data.

THIS. I'd say at least half of the time I went to retrieve a good revision of a file from backup, it was already too late and the backup was trash as well.

Re: Amazon Glacier

#325

Post by tedunangst »

zach wrote: post #28

Amazon Glacier is an extremely low-cost, pay-as-you-go storage service that can cost as little as $0.01 per gigabyte per month. What would be absolutely fascinating is a pay-before-you-go storage service — data cryonics. Paying $12 to store a gigabyte of data for 100 years seems like a pretty intriguing deal as we emerge from an era of bit rot.

If you really wanted to preserve something, I'd consider printing it on gold leaf. One gram of gold turns into about 4 sq ft, and assuming we can print at 1200dpi, we'd have 4 x 144 x 1200 x 1200 bits, or about 100 megabytes. So you'd need about ten grams, at a price of $530 or so, plus storage. Though you could just bury it in your backyard.

Re: Amazon Glacier

#326

Post by klint »

lukebaker wrote: post #285
tc wrote: post #222

Beware that retrieval fee! The retrieval fee for 3TB could be as high as $22,082 based on my reading of their FAQ [1]. It's not clear to me how they calculate the hourly retrieval rate. Is it based on how fast you download the data once it's available, how much data you request divided by how long it takes them to retrieve it (3.5-4.5 hours), or the size of the archives you request for retrieval in a given hour? This…

According to this post[1] they charge based on how long it takes them to retrieve the data. The hourly retrieval rate would be the amount of data you requested divided by how long it takes them to retrieve it (3.5 - 4.5 hours). If it takes them 4 hours to retrieve your 3TB, then your peak hourly retrieval rate would be 768GB / hour (3072 GB / 4 hours). Your billable hourly retrieval rate would be 768GB - 1.28GB (3072…

A spokesperson for AWS confirmed this for me for an article [1] I wrote for Wired: "For a single request the billable peak rate is the size of the archive, divided by four hours, minus the pro-rated 5% free tier."

[1] http://www.wired.com/wiredenterprise/2012/08/glacier/

Re: Amazon Glacier

#327

Post by MichaelGG »

TylerE wrote: post #280
digeridoo wrote: post #272

Earlier quoted context omitted.

At 10Gbps I can retrieve 3TB within an hour.

If you can afford 10Gbps to the internet, $22k is probably chump change.

Afford a 10Gbps connection? You can buy 1Gbps transit for under $1/Mbps, and much less at 10Gbps. So, with a monthly bill of, say, $5K, for the 10Gbps IP, $22k is not quite "chump change".

Re: Amazon Glacier

#328

Post by tomkarlo »

ricardobeat wrote: post #318
tomkarlo wrote: post #309

Earlier quoted context omitted.

Having a multi-hour delay in retrieval lets them move it into their off-peak hours. Since their bandwidth costs are probably calculated off their peak usage, a service that operates entirely in the shadow of that peak has little to no incremental cost to them.

If every request is delayed by the same amount, then it just moves the peak. Otherwise it just flattens/spreads it around peak_length + 4 .

I'm not talking load so much as network traffic. Companies like Amazon and Google have huge peak hour outbound traffic during US waking hours, and then a huge dip during off hours. If they can push more of the traffic into those off hours they can make the marginal cost of the bandwidth basically zero.

So if you make a request at peak hours (say 12 noon ET), they just make you wait until 11 PM ET to start downloading, shifting all that bandwidth off their peak.

Even if you're just "flattening" the peak, when it comes to both CPU and bandwidth, that's a major cost reduction since their cost is driven by peak usage and not average usage in most cases.

Re: Amazon Glacier

#329

Post by xyzzyz »

TylerE wrote: post #280
digeridoo wrote: post #272

Earlier quoted context omitted.

At 10Gbps I can retrieve 3TB within an hour.

If you can afford 10Gbps to the internet, $22k is probably chump change.

You can afford 1Gbps for as little as $70/mo. Currently only in Kansas City, but it will probably expand to other cities soon.

Re: Amazon Glacier

#330

Post by revertts »

sreitshamer wrote: post #160
ghshephard wrote: post #36

I'm a long time user of backblaze, and I'm a big fan of the product - it does a great job of always making sure my working documents are backed up, particularly when I'm traveling overseas, and my laptop is more vulnerable to theft or damage. With that said - Backblaze is optimized for working documents - and the default "exclusion" list makes it clear they don't want to be backing up your "wab~,vmc,vhd,vo1,vo2,vsv,v…

I'm looking into supporting Glacier in Arq. It sure is cheap -- $10/month for a terabyte.

I'll buy Arq immediately when Glacier gets added
Post reply on HN