Live data from Hacker News

Amazon Glacier

aws.amazon.com

221–230 of 393 posts

Re: Amazon Glacier

#221
post #90

I'm a long time user of backblaze, and I'm a big fan of the product - it does a great job of always making sure my working documents are backed up, particularly when I'm traveling overseas, and my laptop is more vulnerable to theft or damage. With that said - Backblaze is optimized for working documents - and the default "exclusion" list makes it clear they don't want to be backing up your "wab~,vmc,vhd,vo1,vo2,vsv,v…

Home-use is probably the only situation Glacier is good for backup though. A home user is fine with a 3.5-4 hour window before their backup becomes available for download (as it will probably take them days to download it anyway). In a corporate environment, I don't want to wait around for 3.5-4 hours before my data even becomes available for restore in a disaster recovery situation. Seems good for archive-only in a…

This would work great as a disaster recovery solution for the museum I work at. Not just for home-use.

Re: Amazon Glacier

#222
Beware that retrieval fee!

The retrieval fee for 3TB could be as high as $22,082 based on my reading of their FAQ [1].

It's not clear to me how they calculate the hourly retrieval rate. Is it based on how fast you download the data once it's available, how much data you request divided by how long it takes them to retrieve it (3.5-4.5 hours), or the size of the archives you request for retrieval in a given hour?

This last case seems most plausible to me [6] -- that the retrieval rate is based solely on the rate of your requests.

In that case, the math would work as follows:

After uploading 3TB (3 * 2^40 bytes) as a single archive, your retrieval allowance would be 153.6 GB/mo (3TB * 5%), or 5.12 GB/day (3TB * 5% / 30). Assuming this one retrieval was the only retrieval of the day, and as it's a single archive you can't break it into smaller pieces, your billable peak hourly retrieval would be 3072 GB - 5.12 GB = 3066.88 GB.

Thus your retrieval fee would be 3066.88 * 720 * .01 = $22081.535 (719x your monthly storage fee).

That would be a wake-up call for someone just doing some testing.

--

[1] http://aws.amazon.com/glacier/faqs/#How_will_I_be_charged_wh...

[2] After paying that fee, you might be reminded of S4: http://www.supersimplestorageservice.com/

[3] How do you think this interacts with AWS Export? It seems that AWS Export would maximize your financial pain by making retrieval requests at an extraordinarily fast rate.

[(edit) 4] Once you make a retrieval request the data is only available for 24 hours. So even in the best case, that they charge you based on how long it takes you to download it (and you're careful to throttle accurately), the charge would be $920 ($0.2995/GB) -- that's the lower bound here. Which is better, of course, but I wouldn't rely on it until they clarify how they calculate. My calculations above represent an upper bound ("as high as"). Also note that they charge separately for bandwidth out of AWS ($368.52 in this case).

[(edit) 5] Answering an objection below, I looked at the docs and it doesn't appear that you can make a ranged retrieval request. It appears you have to grab an entire archive at once. You can make a ranged GET request, but that only helps if they charge based on the download rate and not based on the request rate.

[(edit) 6] I think charging this way is more plausible because they incur their cost during the retrieval regardless of whether or how fast you download the result during the 24 hour period it's available to you (retrieval is the dominant expense, not internal network bandwidth). As for the other alternative, charging based on how long it takes them to retrieve it would seem odd as you have no control over that.

Re: Amazon Glacier

#223

I'm a long time user of backblaze, and I'm a big fan of the product - it does a great job of always making sure my working documents are backed up, particularly when I'm traveling overseas, and my laptop is more vulnerable to theft or damage. With that said - Backblaze is optimized for working documents - and the default "exclusion" list makes it clear they don't want to be backing up your "wab~,vmc,vhd,vo1,vo2,vsv,v…

I'm looking into supporting Glacier in Arq. It sure is cheap -- $10/month for a terabyte.

Arq is great backup software. I would view inclusion of Glacier support as a worthy paid upgrade.

Arq is the pinnacle of my rather large backup pyramid which also includes: Dropbox, Superduper, Crashplan, rsync, SVN/GIT and more.

Re: Amazon Glacier

#224
post #34
post #24

Storage experts: I'd love to know more about what might be backing this service. What kind of system has Amazon most likely built that takes 3-4 hours to perform retrieval? What are some examples of similar systems, and where are they installed?

Typically they are tiered. There'll be a near-line HDD array. This is for the recent content and content they profile as being common-access. Then there'll be a robotic tape library. Any restore request will go in a queue annd when an arm-tapedrive becomes free they'll seek to the data and read it into the HDD array. Waiting for a slot with the robot arm - tape drive is what will take 4 hours. EMC(kinda), Fujitsu etc…

Do you think there is redundancy baked into the tape drive system?

Re: Amazon Glacier

#226
post #214
post #197

Earlier quoted context omitted.

CrashPlan shows your encryption key in plain text. Doesn't that worry you?

Where does it show that? And are you sure we are talking about the encryption key, not the account password?

Settings / Security / Archive Encryption / 448-bit encryption with custom 448-bit key

Re: Amazon Glacier

#227
I'm with Atlantic.net cloud [AWS competitor, full disclosure]; the price point for storage is great but retrieval seems expensive -- perhaps retrieval is very rare its offset by the savings on the storage. I know you can mail in drives for storage, can you have them mail you drives for retrieval? (for Glacier specifically)

Also, prior comments made mention they were using some sort of robotic tape devices, but according to this blog:

http://www.zdnet.com/amazon-launches-glacier-cloud-storage-h...

Its using "commodity hardware components". So, thats why I thought maybe they are loss-leadering on the storage and making up on the retrieval prices?

Its definately a interesting product and I love how there's a reason they called it Glacier. AMZN is a wild boar going after everyone!

Re: Amazon Glacier

#228
post #197
post #80

Earlier quoted context omitted.

What's even more important, you will be able to encrypt your backups without having to disclose the encryption key in case you ever need to restore (client-side encryption and decryption). This is not the case with Backblaze, which is why I switched to CrashPlan — but I'm still looking for other solutions.

CrashPlan shows your encryption key in plain text. Doesn't that worry you?

CrashPlan defaults to using the account password as the encryption password.

However, you can also secure the encryption with a password not associated with the account. Or even provide your own 448-bit key. If you do either of these options, CrashPlan support will not be able to help you

This setup allows CrashPlan to easily help non-technical home users, while allowing technically savvy users to securely hang themselves with their own encryption.

Re: Amazon Glacier

#229
post #197
post #80

Earlier quoted context omitted.

What's even more important, you will be able to encrypt your backups without having to disclose the encryption key in case you ever need to restore (client-side encryption and decryption). This is not the case with Backblaze, which is why I switched to CrashPlan — but I'm still looking for other solutions.

CrashPlan shows your encryption key in plain text. Doesn't that worry you?

You'd prefer that they lie to you and pretend it's not stored in the clear? If you don't have to type it in every time a backup runs, it's in the clear - everything else is just window dressing.

Re: Amazon Glacier

#230
post #67

Earlier quoted context omitted.

Virgin (200+ businesses operating or having operated, under the Virgin brand, ranging from infrastructures - trains, airlines - to records, banking, bridal saloons under the name Virgin Bride...). Mitsubishi and Samsung springs to mind as two of the best known ones internationally where their brands are known in multiple markets internationally, though many of their businesses are less known outside Asia (e.g. Mitsub…

Like you say, many of the big Asian companies are prime examples - Yamaha manufactures Motorcycles and Synthesizers, seems like a good combination!

Musical instruments (especially wind instruments) and motorcycles share a lot of similar design principles. Ever compared a saxophone to a WWII-vintage motorcycle engine?
Post reply on HN