Can anyone make sense of the retrieval fees? Seems like the most confusing thing ever. If I'm storing 4TB and one day I want to restore all 4TB, how much is it going to cost me?
Amazon Glacier
81–90 of 393 posts
Re: Amazon Glacier
#82that's certainly interesting. as there will be migration from s3 to glacier, it would be nice if tarsnap had an option to store only the (say) last week in s3 (with .3$/gb/month) and the rest in glacier (with, say, .03$/gb/month). that would certainly be very nice. cperciva, what do you think?
I can't see any way for Tarsnap to use this right now. When you create a new archive, you're only uploading new blocks of data; the server has no way of knowing which old blocks of data are being re-used. As a result, storing any significant portion of a user's data in Amazon Glacier would mean that all archive extracts would need to go out to Glacier for data... Also, with Tarsnap's average block size (~ 64 kB uncom…
i understand. i hoped tarsnap knew what blocks (do not) get reused.
it's unfortunate, because some backups happen to just lie around for very long. it would be nice to take advantage of (the low cost of) glacier for that.
that said, if it's not possible with tarsnap now, it's not possible now. :D. if you find a satisfying possibility to incorporate it in the new backend(s) design (if that's fixable in the backend(s) alone), i'd surely be pleased.
Re: Amazon Glacier
#83Earlier quoted context omitted.
I can't see any way for Tarsnap to use this right now. When you create a new archive, you're only uploading new blocks of data; the server has no way of knowing which old blocks of data are being re-used. As a result, storing any significant portion of a user's data in Amazon Glacier would mean that all archive extracts would need to go out to Glacier for data... Also, with Tarsnap's average block size (~ 64 kB uncom…
While I have no idea how you would fit it in your current infrastructure, I certainly see a (BIG) use-case for: I have this 100 GB, store it somewhere safe (in glacier), I won't need it for the next year (unless my house burns down). I agree that is a bit different from ongoing daily backups with changes, but its also not THAT different from a customer perspective. That it doesn't fit with how you store blocks on the…
Oh, I absolutely agree that Glacier has lots of great use cases. I wish Tarsnap was able to make good use of it.
Re: Amazon Glacier
#84Excellent! Is there any good open source client / backup application for this? I would start using it immediately. I'm currently using ridiculously expensive backup solution.
Re: Amazon Glacier
#85This is fantastic. I've long searched for a solution like that. This is really suitable for a remote backup that only needs to be accessed if something really bad happens (i.e. a fire breaking out, etc). I'm a lone entrepreneur, so I do have backup hard disks here, but being able to additionally save this data in the cloud is great. I'm often creating pretty big media assets, so Dropbox doesn't necessarily offer enou…
> Glacier would be $10 a month for 1 terabyte. Fantastic.
+ the $120 or so per TB to transfer it outside of AWS if you need the whole thing back as fast as possible. Still likely to be very cheap as long as you treat it as a disaster recovery backup, though. Will definitively consider it.
(an alternative for you is a service like Crashplan, which also allows you very easy access to past file revisions via a java app and can be very cheap and also allow "peer to peer" backups with your friends/family; the downside with Crashplan is that it can be slow to complete a full initial backup to their servers or to get fully backed up again if you move large chunks of data around)
Re: Amazon Glacier
#86One recurrent issue with Amazon services is that they charge in US$ and currently do not accept euros. European banks charge an arm and a leg for micropayment conversions: last time I got a bill from AWS for 0.02$ it ended up costing me 20 euros or so. Pretty much kills the deal. One solution could be to pre-pay 100$ on an account and let them debit as needed.
That doesn't happen in Greece. My debit card charges 0.02ish Eurocents for that.
Re: Amazon Glacier
#87One recurrent issue with Amazon services is that they charge in US$ and currently do not accept euros. European banks charge an arm and a leg for micropayment conversions: last time I got a bill from AWS for 0.02$ it ended up costing me 20 euros or so. Pretty much kills the deal. One solution could be to pre-pay 100$ on an account and let them debit as needed.
That's really odd, I've never heard of fees that high - in the UK I can use my credit cards with Amazon and only pay a few pennies in transaction fees. Have a look at the CaxtonFX Dollar Traveller, you need to load it with $200 but then there's no transaction fees I think, except a slightly worse exchange rate..
Sure there are workarounds. But why should I have to jump through hoops when my account at Amazon is already entrusted with a European credit card and merrily charging euros for all other goods?
Re: Amazon Glacier
#88This is a really good offering for media that you typically will keep locally for instant access, yet you want to have an off-site backup in a way that lives for a very long time. Dropbox should work here, but it's simply too expensive. My photo library is 175GB. That isn't excessive when considering I store the digital negatives and this represents over a decade. I don't mind not being able to access it for a few ho…
sigh Dropbox should not be used as a backup system. A system that synchronizes live should never get that role, except if they can guarantee that old data is never overwritten, new is always appended. This is not the case with dropbox - I've experienced multiple scary occurrences of old versions going to the nirvana with certain user actions. In certain cases the old data just appears to be gone, in other cases the w…
No sigh was necessary, I understand. I even mentioned that I have a NAS and off-site backup... so either you didn't read it all or you stopped the moment you encountered the word "Dropbox" and started typing.
That said, people DO use DropBox as backup.
If you take a walk around the British Library and asked every PhD student working there how they "Backup" their research and work in progress, I bet every single person who believes that they have a backup will say "Dropbox", and the only exceptions will be a few who don't really have a backup.
I know that because I ensured my girlfriend does have a real backup solution in place that is tested. Not one of her peers seems to.
DropBox is used for backup because they've made file sync so damn easy that most people can be convinced that if a file exists in many places, it is backed-up.
My whole point is that now storage for long term backup is priced in a way that is affordable to most, that consumer services may emerge that offer true backup to consumers and can successfully migrate people from lesser solutions (DropBox, stacks of CD-ROMs, etc).
One of the things about backup is that it needs to be easy. Currently the size and cost of backups make it expensive, and the only way to reduce the cost makes it difficult (HDD local copies stored at a friends' house for example).
By reducing the cost, perhaps we can finally increase the ease... and then a day may come in which most people have a real backup solution.
Re: Amazon Glacier
#89Earlier quoted context omitted.
Perhaps DropBox et al will be able to make use of this Infrastructure to lower the costs of their service...
How do you think they could use this service? I can't think of a use case for them.
They store the last 30 days worth of versions of every file you modify. Dropbox could keep versions from the last 10 days in S3 but move the rest to Glacier. Restores for older versions wouldn't be instant, but if S3 storage is a nontrivial expense for them moving the bulk of previous versions to Glacier would cut down on costs.
Re: Amazon Glacier
#90I'm a long time user of backblaze, and I'm a big fan of the product - it does a great job of always making sure my working documents are backed up, particularly when I'm traveling overseas, and my laptop is more vulnerable to theft or damage. With that said - Backblaze is optimized for working documents - and the default "exclusion" list makes it clear they don't want to be backing up your "wab~,vmc,vhd,vo1,vo2,vsv,v…
Home-use is probably the only situation Glacier is good for backup though.
A home user is fine with a 3.5-4 hour window before their backup becomes available for download (as it will probably take them days to download it anyway).
In a corporate environment, I don't want to wait around for 3.5-4 hours before my data even becomes available for restore in a disaster recovery situation.
Seems good for archive-only in a corporate environment (as the name implies).