Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS Tape Gateway

aws.amazon.com

Post reply on HN
Print view View original

41–50 of 139 posts

Re: AWS Tape Gateway

#41

Post by WrtCdEvrydy »

dzdt wrote: post #2

As always the problem seems to be the 'Hotel California' issue: you can check out but you can never leave. Once you have massive data in AWS there is no efficient and affordable solution to move that data back out; you are locked in forever subject to whatever future terms Amazon may chose to impose.

> Once you have massive data in AWS there is no efficient and affordable solution to move that data back out; you are locked in forever subject to whatever future terms Amazon may chose to impose.

You can have your shit packed into a snowball and shipped to you.

Re: AWS Tape Gateway

#42

Post by floren »

thomaslord wrote: post #35
floren wrote: post #27

Earlier quoted context omitted.

Backblaze B2 would cost you about $10/mo to store 2TB. I've been using them for years for a few tens of gigabytes worth of documents and photos. https://www.backblaze.com/b2/cloud-storage-pricing.html edit: it integrates with TrueNAS (formerly FreeNAS) so it's been pretty much set-and-forget, but I can check in and see my stuff through their management interface. I do encrypt my docs before uploading, which TrueNAS a…

Have you had any luck with picking and choosing what gets backed up? I have a bunch of smaller personal files I'd love to sync to B2 (photos, backups, etc.) but I also have 7TB+ of linux ISOs that don't really need to be backed up because I can easily download them again. I tried to exclude the folders I don't want to back up, but I haven't had any luck.

It's been a long time since I set it up, but I believe each cloud sync task will sync one directory. So I've got two tasks: one that syncs /mnt/main/media/Photos in PUSH/COPY mode, and one that syncs /mnt/main/Documents in PUSH/SYNC mode.

You should be able to accomplish the same, but you may need to either re-organize your stuff or set up multiple tasks.

edit: for clarification, "/mnt/main/media" contains various other subdirectories of Linux ISOs etc., which is why I push /mnt/main/media/Photos specifically.

Re: AWS Tape Gateway

#43

Post by jwineinger »

jquaint wrote: post #31
chrisshroba wrote: post #24

Tangent: What's the cheapest way to backup a few TB of personal data these days, pricing based on the premise that I probably will never need to retrieve it (due to local backups as well), but I don't want to pay thousands if I do have to (hundreds would be okay). Glacier Deep Archive?

I recently set up this: https://github.com/vsespb/mt-aws-glacier Found it a lot easier than rolling my own AWS Glacier solution.

That's quite an old (perl) codebase. Did you have any issues?

Re: AWS Tape Gateway

#44

Post by robocat »

dzdt wrote: post #2

As always the problem seems to be the 'Hotel California' issue: you can check out but you can never leave. Once you have massive data in AWS there is no efficient and affordable solution to move that data back out; you are locked in forever subject to whatever future terms Amazon may chose to impose.

“Transfer up to 100 PB per Snowmobile, a 45-foot-long ruggedized shipping container pulled by a semi-trailer truck.” - I’m guessing usually used to migrate corporate information into AWS, but can also be used for data export. They advertise Exabytes (using more than one rig I guess). https://aws.amazon.com/snowmobile/

Re: AWS Tape Gateway

#45

Post by jimt1234 »

dzdt wrote: post #2

As always the problem seems to be the 'Hotel California' issue: you can check out but you can never leave. Once you have massive data in AWS there is no efficient and affordable solution to move that data back out; you are locked in forever subject to whatever future terms Amazon may chose to impose.

AWS will do anything to get your data into their cloud. I realized this a few years ago when Snowmobile was released: https://aws.amazon.com/snowmobile

Re: AWS Tape Gateway

#46

Post by howeyc »

chrisshroba wrote: post #24

Tangent: What's the cheapest way to backup a few TB of personal data these days, pricing based on the premise that I probably will never need to retrieve it (due to local backups as well), but I don't want to pay thousands if I do have to (hundreds would be okay). Glacier Deep Archive?

Scaleway Glacier

Re: AWS Tape Gateway

#47

Post by jedberg »

glasss wrote: post #10

My first and only experience with tape backups was in 2014 working with the BBB of Chicago. They were using tape backups, and there I learned that almost no company that small should be using tapes. Unless you have one or two people dedicated to handling that plus whatever other backup solutions you have in place, it won't end well.

Interesting. I wonder if tapes got more complicated or if everything else just got easier.

The last time I dealt with tapes was in the 1990s, and it was dead simple. We used tape backups at a company with only 150 people. I was the most junior admin so I had to be the tape monkey, but it was easy. Just run backups regularly, change the tape every couple of days and label it, ship a stack to the offsite storage and request the old stack back, and then do a test restore each time a stack came back.

The whole thing took maybe two hours a week.

Re: AWS Tape Gateway

#48

Post by kcb »

jurassic wrote: post #23

> Tape Gateway stores virtual tapes in Amazon S3, Amazon S3 Glacier Flexible Retrieval, and Amazon S3 Glacier Deep Archive, protected by 99.999999999% of durability. That’s a lot of 9s.

I hate it. Once the number gets so small beyond comprehension it’s hard to believe the validity. It reminds me of the Feynman’s statements during the Space Shuttle Challenger disaster.

> Feynman was disturbed by two aspects of this practice. First, NASA management assigned a probability of failure to each individual bolt, sometimes claiming a probability of 1 in 10^8, i.e. one in one hundred million. Feynman pointed out that it is impossible to calculate such a remote possibility with any scientific rigor.

https://en.wikipedia.org/wiki/Rogers_Commission_Report

Unless S3 can survive nuclear war or an asteroid hitting the earth, I don’t buy it.

Re: AWS Tape Gateway

#49

Post by logifail »

thedougd wrote: post #7
SCdF wrote: post #5

Dumb question, and I'm guessing this is a "if you don't know it's not for you" situation, but what is the point of a virtual tape? Isn't the point of a tape that it's not virtual? Or is this more replicating tape software apis (WINE/proton style) so you can get rid of physical tapes (because you no longer care about their physicality) without having to change your backup strategy?

It's the latter. In a large enterprise, the backup configuration can be wildly complicated, a matrix of systems, schedules, slas, etc. Just reconfigure your backup software to use this new virtual tape device and you're on your way.

> Just reconfigure your backup software to use this new virtual tape device and you're on your way

Isn't the whole point of tape that it is a physical thing, and may be taken offsite in a truck / stored in a vault, and all that jazz.

"Just reconfiguring" your backup software sounds like a business might just bypass all that without necessarily realising the consequences.

Then "we got hacked and our local backups are gone" -> "restore from offsite tape" -> "oh, actually there are no tapes, it's all the cloud now" -> ... ??

Re: AWS Tape Gateway

#50

Post by aftbit »

jurassic wrote: post #23

> Tape Gateway stores virtual tapes in Amazon S3, Amazon S3 Glacier Flexible Retrieval, and Amazon S3 Glacier Deep Archive, protected by 99.999999999% of durability. That’s a lot of 9s.

Does than mean that you should expect to lose 1 byte for every 100 GB that you store?
Post reply on HN