Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Glacier

aws.amazon.com

Post reply on HN
Print view View original

281–290 of 393 posts

Re: Amazon Glacier

#281

Post by st3fan »

tc wrote: post #222

Beware that retrieval fee! The retrieval fee for 3TB could be as high as $22,082 based on my reading of their FAQ [1]. It's not clear to me how they calculate the hourly retrieval rate. Is it based on how fast you download the data once it's available, how much data you request divided by how long it takes them to retrieve it (3.5-4.5 hours), or the size of the archives you request for retrieval in a given hour? This…

> It's not clear to me how they calculate the hourly retrieval rate.

Probably based on the speed and the number of arms that the robot has that will grab the right tapes for you :-)

I'm not joking.

Re: Amazon Glacier

#282

Post by ironchef »

TylerE wrote: post #280
digeridoo wrote: post #272

Earlier quoted context omitted.

At 10Gbps I can retrieve 3TB within an hour.

If you can afford 10Gbps to the internet, $22k is probably chump change.

I would hope he's talking about direct connect (http://aws.amazon.com/directconnect/) as otherwise, you're correct.

Re: Amazon Glacier

#283

Post by digeridoo »

TylerE wrote: post #280
digeridoo wrote: post #272

Earlier quoted context omitted.

At 10Gbps I can retrieve 3TB within an hour.

If you can afford 10Gbps to the internet, $22k is probably chump change.

10Gbps EC2 instances start at $0.742/hour. Welcome to the cloud. ;-)

I assume the cost is in retrieval though and counted per the Job Creation API, regardless of whether and how quickly you download the data.

but you're right that the 3TB/hour use-case is very hypothetical. Internet archival is just not suitable for those kind of volumes. I think the point OP was making that mistakes like using archives that are too large, or requesting many at once could cost you a lot.

Re: Amazon Glacier

#284

Post by christiannaths »

sreitshamer wrote: post #160
ghshephard wrote: post #36

I'm a long time user of backblaze, and I'm a big fan of the product - it does a great job of always making sure my working documents are backed up, particularly when I'm traveling overseas, and my laptop is more vulnerable to theft or damage. With that said - Backblaze is optimized for working documents - and the default "exclusion" list makes it clear they don't want to be backing up your "wab~,vmc,vhd,vo1,vo2,vsv,v…

I'm looking into supporting Glacier in Arq. It sure is cheap -- $10/month for a terabyte.

Please!!! I'm a very happy Arq customer, this update would be awesome!

Re: Amazon Glacier

#285

Post by lukebaker »

tc wrote: post #222

Beware that retrieval fee! The retrieval fee for 3TB could be as high as $22,082 based on my reading of their FAQ [1]. It's not clear to me how they calculate the hourly retrieval rate. Is it based on how fast you download the data once it's available, how much data you request divided by how long it takes them to retrieve it (3.5-4.5 hours), or the size of the archives you request for retrieval in a given hour? This…

According to this post[1] they charge based on how long it takes them to retrieve the data. The hourly retrieval rate would be the amount of data you requested divided by how long it takes them to retrieve it (3.5 - 4.5 hours).

If it takes them 4 hours to retrieve your 3TB, then your peak hourly retrieval rate would be 768GB / hour (3072 GB / 4 hours). Your billable hourly retrieval rate would be 768GB - 1.28GB (3072 * .05 / 30 / 4 hours).

Total retrieval fee: 766.72 * 720 * .01 = $5520.38 (~180x your monthly storage fee)

The pricing appears to not be optimized for retrieving all your data in one fell swoop. This particular example appears to be a worst case scenario for restoration because you haven't split up your data into multiple archives (doing so would allow you to reduce your peak hourly retrieval by spacing out your requests for each archive) and you want to restore all your data (the free 5% of your data stored doesn't help as much when you want to restore all your data).

[1] https://forums.aws.amazon.com/message.jspa?messageID=374065#...

Re: Amazon Glacier

#286

Post by DASD »

st3fan wrote: post #281
tc wrote: post #222

Beware that retrieval fee! The retrieval fee for 3TB could be as high as $22,082 based on my reading of their FAQ [1]. It's not clear to me how they calculate the hourly retrieval rate. Is it based on how fast you download the data once it's available, how much data you request divided by how long it takes them to retrieve it (3.5-4.5 hours), or the size of the archives you request for retrieval in a given hour? This…

> It's not clear to me how they calculate the hourly retrieval rate. Probably based on the speed and the number of arms that the robot has that will grab the right tapes for you :-) I'm not joking.

Hopefully, they've also calculated the on-call response time for the tape operator making the drive in to work(or to put the console game on pause and walk over DC). Unless they've come a long ways, robot/library drive mechanisms/belts often need adjustment. Besides, someone has to pick up the LTOs that slipped from the gripper.

Re: Amazon Glacier

#287

Post by res0nat0r »

st3fan wrote: post #281
tc wrote: post #222

Beware that retrieval fee! The retrieval fee for 3TB could be as high as $22,082 based on my reading of their FAQ [1]. It's not clear to me how they calculate the hourly retrieval rate. Is it based on how fast you download the data once it's available, how much data you request divided by how long it takes them to retrieve it (3.5-4.5 hours), or the size of the archives you request for retrieval in a given hour? This…

> It's not clear to me how they calculate the hourly retrieval rate. Probably based on the speed and the number of arms that the robot has that will grab the right tapes for you :-) I'm not joking.

Based on the ZDNet article linked elsewhere on the comments, this system does not use any tape at all. It is all commodity hardware and hard drives, pretty much in line with the design of the rest of the services from AWS.

Re: Amazon Glacier

#288

Post by moontear »

josephagoss wrote: post #142
moontear wrote: post #129

Crashplan is still cheaper for storage larger than 400GB. Crashplan+ Unlimited is USD 2.92/month if you take the 4 year package. When I upload 300GB to Amazon and pay 0.01 * 300 = USD 3/month. Amazon would be even more expensive for larger amounts of data. Is there some fine print I'm missing with Crashplan unlimited?

Also remember its free to recover your entire Crashplan archive (I have 500GB with them). If you wanted to recover 500GB with Glacier it would cost $200 @10MB/s (according to someones calculation further down) You have to pay for retrieval

Not totally true, you have to pay for retrieval for 1GB and upwards per month: http://aws.amazon.com/glacier/#pricing as well as 5cent per upload/retrieval request.

Re: Amazon Glacier

#289

Post by legoforte »

sreitshamer wrote: post #160
ghshephard wrote: post #36

I'm a long time user of backblaze, and I'm a big fan of the product - it does a great job of always making sure my working documents are backed up, particularly when I'm traveling overseas, and my laptop is more vulnerable to theft or damage. With that said - Backblaze is optimized for working documents - and the default "exclusion" list makes it clear they don't want to be backing up your "wab~,vmc,vhd,vo1,vo2,vsv,v…

I'm looking into supporting Glacier in Arq. It sure is cheap -- $10/month for a terabyte.

Wow, yes, please do! I've never used Arq, and at the S3 prices it doesn't make sense on top of my Dropbox. But with Glacier, I'd love to be able to back up my entire HDD off-site for $5/month, and Arq looks like a good way to do that.

Re: Amazon Glacier

#290

Post by ghshephard »

rsync wrote: post #265
tc wrote: post #222

Beware that retrieval fee! The retrieval fee for 3TB could be as high as $22,082 based on my reading of their FAQ [1]. It's not clear to me how they calculate the hourly retrieval rate. Is it based on how fast you download the data once it's available, how much data you request divided by how long it takes them to retrieve it (3.5-4.5 hours), or the size of the archives you request for retrieval in a given hour? This…

This was interesting to wake up to this morning ... Right now we sell 10TB blocks for $9500/year[1]. This works out to 7.9 cents/GB, per month, so 7.9x the glacier pricing. However, our pricing model is much simpler, as there is no charge at all for bandwidth/transfer/usage/"gets", so the 7.9 cents is it. 7.9x is a big multiplier. OTOH, users of these 10TB blocks get two free instances of physical data delivery (mail…

I've spent several hours reading about this and talking with colleagues, reading the (really great) HN threads on the topic and doing a bunch of math - and I've come to the conclusion that rsync.net/backblaze/tarsnap/crashplan probably don't have too much to worry about for _most_ use cases.

The wonky pricing on retrieval makes this inordinately complex to price out for the average consumer who will be doing restores of large amounts of data.

The lack of easy consumer flexibility for restores also is problematic for the use case of "Help, I've lost my 150 GB Aperture Library / 1 TB Hard Drive"

The 4 Hour retrieval time makes it a non starter for those of us who frequently recover files (sometime from a different machine) off the website.

The cost is too much for >50 Terabyte Archives - Those users will be likely be doing multi-site Iron Mountain backups on LTO-5 Tapes. After 100 Terabytes, the cost of the drives is quickly amortized and ROI on the tapes is measured in about a month.

The new business model that Amazon may have created overnight though, and beats everyone on price convenience, is "Off-Site Archiving of low-volume low value Documents" - Think Family Pictures. Your average shutterbug probably has on the order of 50 GBytes of photos (give or take) - is it worth $6/year for them to keep a safe offline archive of them? Every single one of those people should be signing up for the first software package that gives them a nice consumer-friendly GUI to backup their picasa/iPhoto/Aperture/Lightroom photo library.

Let's all learn a lesson from [Edit Mat, one t] Honan.

Post reply on HN