Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Glacier

aws.amazon.com

Post reply on HN
Print view View original

171–180 of 393 posts

Re: Amazon Glacier

#171

Post by dtby »

ghshephard wrote: post #134
DeepDuh wrote: post #69

Earlier quoted context omitted.

sigh Dropbox should not be used as a backup system. A system that synchronizes live should never get that role, except if they can guarantee that old data is never overwritten, new is always appended. This is not the case with dropbox - I've experienced multiple scary occurrences of old versions going to the nirvana with certain user actions. In certain cases the old data just appears to be gone, in other cases the w…

The ideal solution is the Quadfecta (is that a word?) - Dropbox for (in my experience) excellent versioning/synchronizing (Never failed me) + Backblaze (or its ilk) for continuous Offline backups + Super Duper (weekly/whenever) - for Image Backups - + Something (Arq?) on top of Glacier for long time off-site-archival. For $50 in software (Arq+SuperDuper), $100 for an external HD, and less than $25/month ($4-backblaze…

> Quadfecta (is that a word?)

In case you happen to read with show-dead on: it's a "superfecta."

Re: Amazon Glacier

#173

Post by kloc »

digeridoo wrote: post #131
kloc wrote: post #120

Amazon should complement this service with data contact centers which are connected to their data centers network. Then people could go to these centers in person and hand over their hard drives full of data for back up. It will be like bank lockers but only digital. At this low price people would want to upload terabytes of data which will be pain to upload/download.

You mean like http://aws.amazon.com/importexport/ ?

Was not aware of this service. Thanks!

Re: Amazon Glacier

#174

Post by noibl »

speleding wrote: post #163
ghshephard wrote: post #116

Earlier quoted context omitted.

Interesting to note - those of us who used Iron Mountain/Data Safe for years - 4 Hours was considered a "Premier Rapid Recovery" service that we paid a lot of money for (as recently as 2003, actually). In a true disaster recovery (Building burned down or is otherwise unavailable) - it usually takes most businesses a week or so just to find new office facilities. But - agreed, there will be some customers for whom Gla…

Yes, that's what I am thinking: a week of backups to S3 and a script that does moves the oldest one from S3 to Glacier. You would rarely need a backup older than a week if you've already got a week of backups. I'm still figuring out if there is a practical way to do this with incremental backups without introducing to much risk that the backup process gets messed up.

"In the coming months, Amazon Simple Storage Service (Amazon S3) plans to introduce an option that will allow you to seamlessly move data between Amazon S3 and Amazon Glacier using data lifecycle policies."

Re: Amazon Glacier

#175

Post by gjm11 »

Jabbles wrote: post #137
gjm11 wrote: post #27

Earlier quoted context omitted.

Which of course means that (if they're telling the truth) the probability of losing your data mostly comes from really big events: collapse of civilization, global thermonuclear war, Amazon being bought by some entity that just wants to melt its servers down for scrap, etc. (Whose probability is clearly a lot more than 10^-11 per year; the big bang was only on the order of 10^10 years ago.)

per object . So although the chance of losing any particular object is tiny, the chance of you losing something is proportional† to the number of objects. Still extremely small. †roughly proportional if you have << 1e11 objects

Yes. Though I bet the real lossage probabilities are dominated by failure events that take out a substantial fraction of all the objects there are, and that happen a lot more often than once per 10^11 years.

Re: Amazon Glacier

#176

Post by g-garron »

DeepDuh wrote: post #69
buro9 wrote: post #41

This is a really good offering for media that you typically will keep locally for instant access, yet you want to have an off-site backup in a way that lives for a very long time. Dropbox should work here, but it's simply too expensive. My photo library is 175GB. That isn't excessive when considering I store the digital negatives and this represents over a decade. I don't mind not being able to access it for a few ho…

sigh Dropbox should not be used as a backup system. A system that synchronizes live should never get that role, except if they can guarantee that old data is never overwritten, new is always appended. This is not the case with dropbox - I've experienced multiple scary occurrences of old versions going to the nirvana with certain user actions. In certain cases the old data just appears to be gone, in other cases the w…

I use a combination of Dropbox and S3.

I have my MacBook and a Linux server linked to my Dropbox account. So changes in my documents are synced to my Linux.

My Linux run three cron-jobs. One daily, one weekly and another monthly. The command is.

s3cmd sync --delete-removed ~/Dropbox/documents/ s3://backup-daily/

There are buckets for weekly and monthly too.

Note: the command is not exactly like that, check the man page.

That way I have backed up all my documents very cheap.

Re: Amazon Glacier

#177

Post by Yrlec »

mik4el wrote: post #170
Yrlec wrote: post #168

This looks awesome! We are currently developing a P2P-based backup solution ( http://degoo.com ) where we are using S3 as fall-back. This will allow us to be much cheaper and I am sure it will enable many other backup providers to lower their prices to.

The S3->Glacier lifecycle-feature must be a welcome thing for you I guess?

Definitely! Hopefully it can enable us to provide a good trade-off between cost and restore time.

Re: Amazon Glacier

#178

Post by nodata »

wladimir wrote: post #158
nodata wrote: post #98

Earlier quoted context omitted.

Duplciity supports S3, so I'd watch it for Glacier support: http://duplicity.nongnu.org/

I wonder if Glacier support in Duplicity will be possible without large changes. AFAIK, duplicity also reads some state from the remote end to determine what to backup (Although it also keeps a local cache of this?). To use glacier, the protocol would have to completely write-only.

Turns out there is less of a need for direct support: https://news.ycombinator.com/item?id=4411649

Re: Amazon Glacier

#179

Post by andyjohnson0 »

arethuza wrote: post #101
zach wrote: post #28

Amazon Glacier is an extremely low-cost, pay-as-you-go storage service that can cost as little as $0.01 per gigabyte per month. What would be absolutely fascinating is a pay-before-you-go storage service — data cryonics. Paying $12 to store a gigabyte of data for 100 years seems like a pretty intriguing deal as we emerge from an era of bit rot.

"Paying $12 to store a gigabyte of data for 100 years" I'm not sure what kind of organisation I'd actually trust to store data for that length of time - a commercial organisation is probably going to be more effective at providing service but what commercial organisation would you trust to provide consistent service for 100 years? A Swiss bank perhaps? Governments of stable countries are obviously capable of this (cl…

There are some pretty old commercial organisations out there. One list [1] has several that have existed for over 1000 years. Whether they would be capable of storing data (or would even be interested in doing so) is hard to say, but there have at least demonstrated that long-term organisational continuity is possible, and that presumably requires some organisational 'memory'.

Having a long history obviously isn't a predictor of future stability. According to the Long Now Foundation site [2], a Japanese company that existed since 578CE went bust in 2007.

[1] http://www.bizaims.com/content/the-100-oldest-companies-worl... [2] http://blog.longnow.org/02008/06/13/the-100-oldest-companies...

Re: Amazon Glacier

#180

Post by ghshephard »

g-garron wrote: post #176
DeepDuh wrote: post #69

Earlier quoted context omitted.

sigh Dropbox should not be used as a backup system. A system that synchronizes live should never get that role, except if they can guarantee that old data is never overwritten, new is always appended. This is not the case with dropbox - I've experienced multiple scary occurrences of old versions going to the nirvana with certain user actions. In certain cases the old data just appears to be gone, in other cases the w…

I use a combination of Dropbox and S3. I have my MacBook and a Linux server linked to my Dropbox account. So changes in my documents are synced to my Linux. My Linux run three cron-jobs. One daily, one weekly and another monthly. The command is. s3cmd sync --delete-removed ~/Dropbox/documents/ s3://backup-daily/ There are buckets for weekly and monthly too. Note: the command is not exactly like that, check the man pa…

One thing to be a little careful of - you are sending all of your data to S3, which is, of course, the backing store for Dropbox.
Post reply on HN