Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS Restored My Account: The Human Who Made the Difference

seuros.com

Post reply on HN
Print view View original

11–20 of 95 posts

Re: AWS Restored My Account: The Human Who Made the Difference

#11

Post by eqvinox »

> "people in positions of leadership, such as my boss, are aware of your blog post and I’ve been tasked with finding out what I can"

Translation: "someone noticed it trending on HN, decided it was bad publicity, and that they should do something about it"

Implication: what mattered was the bad publicity, not the poor support infrastructure. The latter won't change, and the next person with similar problems will get the same runaround, and probably lose their data.

/c (cynic, but I suspect realist)

Re: AWS Restored My Account: The Human Who Made the Difference

#12

Post by qualeed »

Awesome that the author got their stuff back. But...

>My data is back. Not because of viral pressure. Not because of bad PR. [...]

>“I am devastated to read on your blog about the deletion of your AWS data. I did want to reach out to let you know that people in positions of leadership, such as my boss, are aware of your blog post and I’ve been tasked with finding out what I can, and to at least prevent this from happening in the future.”

So, yes, because of bad PR. Or, at least the possibility of the blog blowing up into a bad PR storm. I'm guessing that if there was no blog, the outcome would be different.

Re: AWS Restored My Account: The Human Who Made the Difference

#15

Post by dylan604 »

webstrand wrote: post #9
Ezhik wrote: post #4

I hope the author follows up with how they build out a backup system. No snark intended - it really does seem to me like an interesting set of articles especially for others who might be in a similar situation with keeping everything with one cloud provider.

Honestly it's pretty worrying, I don't understand how other people can talk about "you have to have multiple offsite backups" when each offsite runs about $700 per year for a relatively small amount of data. I can only afford one off-site backup, and if my local copy dies it'll take a couple of weeks for me to restore from AWS because of local internet speed limitations.

Offsite does not have to mean another cloud solution. If you have a copy of data on a thumbdrive or other external media, you can leave it at your parent's house (or other family member). You can keep a copy of it in a safety deposit box. If it's at a family member's house, it's just an excuse to visit on a semi-regular schedule as you swap out with an updated version.

Re: AWS Restored My Account: The Human Who Made the Difference

#16

Post by jacquesm »

One of the questions we always ask of cloud hosted companies what their plan is in case they ever lose their cloud account. Typically this is met with incredulity: nobody has ever lost their AWS account and all it's data? Right? Right???

Well, not normally, no. But it does happen. Not often enough to be a meaningful statistical issue, but if it were to happen to you then a little forethought can turn a complete disaster into a survivable event. If you store all your data 'in the cloud' realize that your account could be compromised, used to store illegal data, be subject to social engineering and lots of other ways that could result in a cloud services provider to protect their brand rather than your data. If - like the author - you are lucky you'll only be down for a couple of days. But for most businesses that's the end of the line, especially if you run a multi-tenant SaaS or something like that. So plan for the worst and hope for the best.

Re: AWS Restored My Account: The Human Who Made the Difference

#18

Post by pavel_lishin »

dylan604 wrote: post #15
webstrand wrote: post #9

Earlier quoted context omitted.

Honestly it's pretty worrying, I don't understand how other people can talk about "you have to have multiple offsite backups" when each offsite runs about $700 per year for a relatively small amount of data. I can only afford one off-site backup, and if my local copy dies it'll take a couple of weeks for me to restore from AWS because of local internet speed limitations.

Offsite does not have to mean another cloud solution. If you have a copy of data on a thumbdrive or other external media, you can leave it at your parent's house (or other family member). You can keep a copy of it in a safety deposit box. If it's at a family member's house, it's just an excuse to visit on a semi-regular schedule as you swap out with an updated version.

Yep. My backups - personal ones, not business - live in three places:

1. A hard drive in a fire safe.

2. An S3 bucket, mediated by Wasabi.

3. My friend's server that lives at his house half a continent away.

It would be nice to have a fourth location that's a physical hard-drive that lives outside of my house, but close enough to drive to for pick-up, but it would mean either paying for a safety deposit box as you mentioned, or hassling a friend once a week as I come to pick it up and deposit it.

Re: AWS Restored My Account: The Human Who Made the Difference

#19

Post by tetha »

svt1234 wrote: post #7
Ezhik wrote: post #4

I hope the author follows up with how they build out a backup system. No snark intended - it really does seem to me like an interesting set of articles especially for others who might be in a similar situation with keeping everything with one cloud provider.

I don't use AWS but I have a server on Linode and I realized that I am in the same situation should Linode go away, even though I tend to have a pretty rigorous backup routine elsewhere. It is something you can easily miss. I am inspired now to dump my databases and rsync the content on a schedule.

Within Europe and if you're not at Hetzner already, Hetzner Storage Boxes[1] are very competitive. 4 Euros per month for a terabyte, throw borg[2] with compression + deduplication at it and it'll take you far. Especially if your databases are not changing that much, then borg can make that terabyte of storage a lot bigger.

1: https://www.hetzner.com/storage/storage-box/

2: https://www.borgbackup.org/

Re: AWS Restored My Account: The Human Who Made the Difference

#20

Post by colmmacc »

eqvinox wrote: post #11

> "people in positions of leadership, such as my boss, are aware of your blog post and I’ve been tasked with finding out what I can" Translation: "someone noticed it trending on HN, decided it was bad publicity, and that they should do something about it" Implication: what mattered was the bad publicity, not the poor support infrastructure. The latter won't change, and the next person with similar problems will get t…

Every week at AWS we have an account protection meeting; it's got the teams who handle fraud and abuse detection, compromised accounts (e.g. when a customer has a security issue on their side), non-payment, forgotten creds, as well as our support team. You'll find the most junior members of those teams, all the way up to VPs, in the same meeting diving into the nitty gritty. Every week.

Disabling a legitimate in-use account is one of our absolute nightmares, and I don't care if it was an account paying $3/month we would be having a review of that with our top level management (including our CEO - Matt Garman) no matter how we found out about it. For us, there is not some acceptable rate of this as a cost of doing business.

Post reply on HN