Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS Tape Gateway

aws.amazon.com

Post reply on HN
Print view View original

71–80 of 139 posts

Re: AWS Tape Gateway

#71

Post by CSMastermind »

zetazzed wrote: post #61

I think I'll point to this as one of the examples of how AWS has been scrappy and successful. (Yes, I'll get like 10 flames on this comment.) They made this nice simple API for S3 that has become a de facto standard. They added lots of features and security controls, and I can easily imagine an architect saying "everyone should just adopt S3, it's so easy!" But huge, paying customers were used to tape libraries so th…

I use AWS at the companies I work for because of their amazing customer service.

I'm fairly critical of Amazon's engineering culture and practices but as a customer I have to admit they are head and shoulders above their competition and frankly most SaaS providers as a whole.

They've secured millions of dollars in revenue from me alone just by being so customer friendly.

Re: AWS Tape Gateway

#72

Post by free-ideas »

shrike wrote: post #20
SCdF wrote: post #5

Dumb question, and I'm guessing this is a "if you don't know it's not for you" situation, but what is the point of a virtual tape? Isn't the point of a tape that it's not virtual? Or is this more replicating tape software apis (WINE/proton style) so you can get rid of physical tapes (because you no longer care about their physicality) without having to change your backup strategy?

The VTL was invented because, at the time (2012ish), none of the largest enterprise backup solutions had a good S3 interface and none supported Glacier. After talking with the backup software vendors (Spectrum, Tivoli, Symantec, Commvault, etc) it became clear that adding another backup target wasn't something we (AWS) could get them to prioritize, for perfectly reasonable reasons. We could (and did) apply pressure v…

VTL has been around since the 90’s https://en.m.wikipedia.org/wiki/Virtual_tape_library.

This is a fantastic example of how broken the patent system is.

This is not an invention it’s a good implementation of a well understood problem.

It is also priced ridiculously - it is at least an order of magnitude more expensive than operating a physical library and remote physical storage that is beyond cyber threat by virtue of it being disconnected.

A real backup is offline and offsite.

Re: AWS Tape Gateway

#73

Post by oneplane »

CSMastermind wrote: post #71
zetazzed wrote: post #61

I think I'll point to this as one of the examples of how AWS has been scrappy and successful. (Yes, I'll get like 10 flames on this comment.) They made this nice simple API for S3 that has become a de facto standard. They added lots of features and security controls, and I can easily imagine an architect saying "everyone should just adopt S3, it's so easy!" But huge, paying customers were used to tape libraries so th…

I use AWS at the companies I work for because of their amazing customer service. I'm fairly critical of Amazon's engineering culture and practices but as a customer I have to admit they are head and shoulders above their competition and frankly most SaaS providers as a whole. They've secured millions of dollars in revenue from me alone just by being so customer friendly.

Yep, that's been my experience as well. AWS (and Amazon as a whole) are bad for the people that work there, but as for their service offerings, it would take GCP and Azure combined to get even close.

Re: AWS Tape Gateway

#74

Post by seized »

chrisshroba wrote: post #24

Tangent: What's the cheapest way to backup a few TB of personal data these days, pricing based on the premise that I probably will never need to retrieve it (due to local backups as well), but I don't want to pay thousands if I do have to (hundreds would be okay). Glacier Deep Archive?

AWS Glacier Deep Archive. It's about $1/TB/month. Retrieve is more expensive, about $90/TB all in.

You can use rclone as one simple tool.

Re: AWS Tape Gateway

#75

Post by harshaw »

jurassic wrote: post #23

> Tape Gateway stores virtual tapes in Amazon S3, Amazon S3 Glacier Flexible Retrieval, and Amazon S3 Glacier Deep Archive, protected by 99.999999999% of durability. That’s a lot of 9s.

Think about it in terms of erasure encoding where you can have enough shards on enough hosts. That's how you get the durability.

Re: AWS Tape Gateway

#76

Post by MisterPea »

oneplane wrote: post #73
CSMastermind wrote: post #71

Earlier quoted context omitted.

I use AWS at the companies I work for because of their amazing customer service. I'm fairly critical of Amazon's engineering culture and practices but as a customer I have to admit they are head and shoulders above their competition and frankly most SaaS providers as a whole. They've secured millions of dollars in revenue from me alone just by being so customer friendly.

Yep, that's been my experience as well. AWS (and Amazon as a whole) are bad for the people that work there, but as for their service offerings, it would take GCP and Azure combined to get even close.

I think if you took a poll of AWS workers you'd find that most enjoy it. The culture leads to a vocal minority who REALLY hate it but my time there was some of the most enjoyable I've ever had.

I've worked with some of the smartest people in the tech field, building products from scratch for billion dollar customers and learning every day.

I've always told people that it's probably the best place you can start out of college.

EDIT: This was back in 2016, I'm not sure how much has changed without Bezos and Jassy at helm of AWS

Re: AWS Tape Gateway

#77

Post by pmw »

WrtCdEvrydy wrote: post #41
dzdt wrote: post #2

As always the problem seems to be the 'Hotel California' issue: you can check out but you can never leave. Once you have massive data in AWS there is no efficient and affordable solution to move that data back out; you are locked in forever subject to whatever future terms Amazon may chose to impose.

> Once you have massive data in AWS there is no efficient and affordable solution to move that data back out; you are locked in forever subject to whatever future terms Amazon may chose to impose. You can have your shit packed into a snowball and shipped to you.

But you pay the same egress rate.

Re: AWS Tape Gateway

#78

Post by logifail »

kube-system wrote: post #67
logifail wrote: post #49

Earlier quoted context omitted.

> Just reconfigure your backup software to use this new virtual tape device and you're on your way Isn't the whole point of tape that it is a physical thing, and may be taken offsite in a truck / stored in a vault, and all that jazz. "Just reconfiguring" your backup software sounds like a business might just bypass all that without necessarily realising the consequences. Then "we got hacked and our local backups are…

AWS's hard disks are a physical thing that is offsite

> AWS's hard disks are a physical thing that is offsite

Taking tapes to a secure offsite location means they're air-gapped from the source data, so even in a situation where an entire network is compromised, remotely-stored tapes can't be wiped/encrypted.

I'm sure AWS has thought about this issue while designing Tape Gateway, but it's not clear to me how it could know whether a request to retrieve and overwrite a virtual tape was legitimate or not.

Re: AWS Tape Gateway

#79

Post by chickenpotpie »

zetazzed wrote: post #61

I think I'll point to this as one of the examples of how AWS has been scrappy and successful. (Yes, I'll get like 10 flames on this comment.) They made this nice simple API for S3 that has become a de facto standard. They added lots of features and security controls, and I can easily imagine an architect saying "everyone should just adopt S3, it's so easy!" But huge, paying customers were used to tape libraries so th…

> damnit, fine we'll pretend to be a tape library

IIRC, they may not be pretending. It's not public knowledge what storage medium AWS glacier is built on, but many speculate it is actually tape.

Re: AWS Tape Gateway

#80

Post by capableweb »

chickenpotpie wrote: post #79
zetazzed wrote: post #61

I think I'll point to this as one of the examples of how AWS has been scrappy and successful. (Yes, I'll get like 10 flames on this comment.) They made this nice simple API for S3 that has become a de facto standard. They added lots of features and security controls, and I can easily imagine an architect saying "everyone should just adopt S3, it's so easy!" But huge, paying customers were used to tape libraries so th…

> damnit, fine we'll pretend to be a tape library IIRC, they may not be pretending. It's not public knowledge what storage medium AWS glacier is built on, but many speculate it is actually tape.

Well, they do call it "virtual tapes", would be weird (maybe even illegal? False/misleading advertisement?) if they ended up storing it on physical tapes anyways.

Edit: found in another thread here, the inventor linked the patent (https://image-ppubs.uspto.gov/dirsearch-public/print/downloa...) which confirms it's virtual tapes, not actual tapes.

Post reply on HN