Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Glacier

aws.amazon.com

Post reply on HN
Print view View original

151–160 of 393 posts

Re: Amazon Glacier

#151

Post by DeepDuh »

Dylan16807 wrote: post #136
DeepDuh wrote: post #126

Earlier quoted context omitted.

Yes, I'm aware of that. This helps, up to the point where you don't notice that something's gone for 'a few days'. Let's say you have your student project's folder shared with two other colleagues. You take a few days off, use your PC for casual browsing, meanwhile your colleagues are working on the project. At the end, one of them (who doesn't quite understand how dropbox works) deletes the files while the other is…

The colleague that was actively using the files doesn't notice them disappear? Can you explain how dropbox has lost history for you in the past?

It doesn't matter whether he notices it or not, you can't count on social factors when it comes to backup systems. He might notice it but then he thinks, oh no problem, it's all in Dropbox anyway, I'll talk to Dylan16807 when he gets back.

I've never had any complete data losses until now, but I've had this situation where the Dropbox cache was the only place to retrieve files twice and it showed me that it can't be trusted this way. I'm certainly not gonna wait until the real deal happens just to have a personal story on how Dropbox can go horribly wrong. I've been shown the possibility and that's enough for me.

As for specifics, the case I mentioned is the one I saw where it can go wrong. IMO the only really save way to use Dropbox without additional backups is if you never share folders and only ever use one system with write-access at the same time - which is not the usual use-case of that product.

Re: Amazon Glacier

#152

Post by ghshephard »

Keyframe wrote: post #144

I'm not sure if cost is right. Each project I work on is approx. 50-60 TB in size (video). Recent one got backed up on 20 LTO 5 tapes times three. That's $600 for tapes per project. Each tape set went to a separate location - two secure ones for about $20/year and one at studio archive for immediate access, if needed. I find this method extremely reliable and it cost ~$700 initially to back all up and virtually non e…

You have a pretty niche (but interesting!) use case. An LTO-5 Tape can store 1.5 Terabytes Raw [1] - Call it 2 Terabytes with a bit of compression (your Video probably doesn't losslessly compress at 2:1). 60 Terabytes requires 30 Tapes - around $15/month to store at Iron Mountain. [2] . The Glacier Charge for 60 Terabytes is $600/month vs $15/month for Tape Storage.

Also - upload/recovery times are problematic when you are talking 10s of terabytes. Right now, the equation is in favor of archiving tapes at that level (Even presuming you store multiple copies for redundancy/safety).

Glacier is for the people wanting to archive in the sub-ten terabyte range - they can avoid the hassle/cost of purchasing tape drives, tapes, software - and just have their online archive.

The needle will move - in 10 years Glacier might make sense for people wanting to store sub 100 Terabytes, and tapes will be for the multi-petabyte people.

[1] http://en.wikipedia.org/wiki/Linear_Tape-Open

[2] http://www.ironmountain.com/Solutions/~/media/9F17511FA1A741...

Re: Amazon Glacier

#153

Post by squidsoup »

gvalkov wrote: post #105

Here's to hoping that duplicity and git-annex could somehow make use of this service. I'm far more optimistic about duplicity support though, as incremental archives seem to fit the glacier storage model much better. A git-annex special remote [1] might turn out to be much more challenging, if at all possible. [1] http://git-annex.branchable.com/special_remotes/

Once there's support for Glacier in boto (https://github.com/boto/boto) I would imagine a duplicity backend would be easy to implement.

Re: Amazon Glacier

#154

Post by bobsy »

arethuza wrote: post #150
wheels wrote: post #145

Earlier quoted context omitted.

> Governments of stable countries are obviously capable of this I don't consider that obvious. I live in Berlin, the capital of what most would consider a stable country, but my apartment (which is even older) has been a part of 5 different countries in the last 100 years (German Empire, Weinmar Republic, Nazi Germany, East Germany and finally, the Federal Republic of Germany).

Sorry, what I meant by "stable" there is countries that have been relatively stable for a few hundred years and seem reasonably likely to continue that integrity for at least a century or so. Of course, predicting future stability is complete guesswork!

We are straying a bit off topic here. I don't think any country has been stable for a "few" hundred years. A few is 3. This is before the US was founded.

I would say a stable country is one which has had a legit democracy for 70 or so years and doesn't share a border with a non-democratic / non-legit-democratic state. These two points suggests its unlikely to have a revolution or be invaded any time soon.

Of course, if you look at the UK governments track record with IT.. you wouldn't trust it. I would say the same with the US, especially in concern with data security.

Re: Amazon Glacier

#155

Post by waterlesscloud »

masterzora wrote: post #51
OoTheNigerian wrote: post #38

What I find fascinating with Amazon's infrastructure push is the the successful 'homonymization' of their brand name Amazon. Amazon simultaneously stands for ecommerce and web infrastructure depending on the context. e.g "Hey I want to host my server".. "Why don't you try Amazon". "Do you know where I can get a fair priced laptop?" "Check Amazon". Is there any other brand that has done this successfully? Edit: I shou…

This actually confused me when I saw the headline. I saw 'Amazon' and, despite having been working with AWS all night, thought ecommerce first and couldn't figure out what they could name 'Glacier'. I was kind-of-not-really hoping it was going to be a new shipping option guaranteed to take a long time. That said, the actual service looks solid.

I'm completely amused that there's an internet service named "Glacier" in such a way that it is a positive connotation!

Re: Amazon Glacier

#156

Post by urza »

I am a bit confused with the pricing of retrieval.

Could some good soul tell me how much would cost to:

Store 150 GB as one big file for 5 years. To this I will add 10 GB (also as one file) every year. And lets say I will need to retrieve the whole archive (original file + additions) at the end of year 2 and 5.

How much will it cost?

Re: Amazon Glacier

#157

Post by Tichy »

kalininalex wrote: post #96
ghshephard wrote: post #36

I'm a long time user of backblaze, and I'm a big fan of the product - it does a great job of always making sure my working documents are backed up, particularly when I'm traveling overseas, and my laptop is more vulnerable to theft or damage. With that said - Backblaze is optimized for working documents - and the default "exclusion" list makes it clear they don't want to be backing up your "wab~,vmc,vhd,vo1,vo2,vsv,v…

With large volumes the real issue is not the storage but the upload speed. I've done some experiments and with the Comcast link (20Mbps down/whatever up), I got 1 Gb/hour upload rate. So, it'll take 11 full days to upload 256 Gb. Or, more realistically, if you do it overnight (8 hours) - the entire month.

But then you can do incremental backups which should be faster. Although I admit I am not sure how to do it with encryption (impossible?).

Re: Amazon Glacier

#158

Post by wladimir »

nodata wrote: post #98
Sami_Lehtinen wrote: post #84

Excellent! Is there any good open source client / backup application for this? I would start using it immediately. I'm currently using ridiculously expensive backup solution.

Duplciity supports S3, so I'd watch it for Glacier support: http://duplicity.nongnu.org/

I wonder if Glacier support in Duplicity will be possible without large changes. AFAIK, duplicity also reads some state from the remote end to determine what to backup (Although it also keeps a local cache of this?). To use glacier, the protocol would have to completely write-only.

Re: Amazon Glacier

#159

Post by jefffoster »

kalininalex wrote: post #96
ghshephard wrote: post #36

I'm a long time user of backblaze, and I'm a big fan of the product - it does a great job of always making sure my working documents are backed up, particularly when I'm traveling overseas, and my laptop is more vulnerable to theft or damage. With that said - Backblaze is optimized for working documents - and the default "exclusion" list makes it clear they don't want to be backing up your "wab~,vmc,vhd,vo1,vo2,vsv,v…

With large volumes the real issue is not the storage but the upload speed. I've done some experiments and with the Comcast link (20Mbps down/whatever up), I got 1 Gb/hour upload rate. So, it'll take 11 full days to upload 256 Gb. Or, more realistically, if you do it overnight (8 hours) - the entire month.

There's also Amazon Direct Connection (http://aws.amazon.com/directconnect/) if you happen to be on the same backbone / data center.

Re: Amazon Glacier

#160

Post by sreitshamer »

ghshephard wrote: post #36

I'm a long time user of backblaze, and I'm a big fan of the product - it does a great job of always making sure my working documents are backed up, particularly when I'm traveling overseas, and my laptop is more vulnerable to theft or damage. With that said - Backblaze is optimized for working documents - and the default "exclusion" list makes it clear they don't want to be backing up your "wab~,vmc,vhd,vo1,vo2,vsv,v…

I'm looking into supporting Glacier in Arq. It sure is cheap -- $10/month for a terabyte.
Post reply on HN