Ok, enough AWS outages to say I'm tired of hearing about low end stuff being flaky.
Nobody ever got fired for using AWS.
maybe except a team at google? ;)
Tell HN: AWS appears to be down again
161–170 of 646 posts
Re: Tell HN: AWS appears to be down again
#162At least HN works.
Re: Tell HN: AWS appears to be down again
#163So why are people not migrating out of us-east-1? Operating in ap-southeast, we weren't that affected by the us-east-1 down time, although our system is reasonably static and doesn't make lots of IAM calls (which seems to be a large SPOF from us-east-1).
Some “global” systems run in us-east1 even if you’re not hosted there a service you depend on might be. Notably: cognito, r53 and the default web UI. (You can work around the webui one I’m told, by passing a different domain instead of just console.aws.amazon.com)
Don't forget about CloudFront, which can only be configured via us-east-1.
Re: Tell HN: AWS appears to be down again
#164Honestly my server at home has more uptime than US-East-1
I should blog about this one day but...
I have a server at OVH (not affiliated to them) which, at this point, I keep only for fun. It has 3162 days of uptime as I type this.
3 162 days. That's 8 years+ of uptime.
Does it have the traffic of Amazon? No.
Is it secure? Very likely not: it's running an old Debian version (Debian 7, which came out in, well, 2013).
It only has one port opened though, SSH. And with quite a hardened SSH setup at that.
I installed all the security patches I could install without rebooting it (so, yes, I know, this means I didn't install all the security patches for some required rebooting).
This server is, by now, a statement. It's about how stable Linux can be. It's about how amazingly stable Debian is. It's also about OVH: at times they had part of their datacenter burn (yup), at times they had full racks that had to be moved/disconnected. But somehow my server never got affected. It may have happened that at one point OVH had connectivity issues but my server went down.
I "gave back" many of my servers I didn't need anymore. But this one I keep just because...
I still use it, but only as an additional online/off-site backup where I send encrypted backups. It's not as if it gets zero use: I typically push backups to it daily.
They're only backups, they're encrypted. Even if my server is "owned" by some bad guys, the damage he could do is limited. Never seen anything suspicious on it though.
I like to do "silly" stuff like that. Like that one time I solve LCS35 by computing for about four years on commodity hardware at home.
I think it's about time I start to do some archeology on that server, to see what I can find. Apparently I installed Debian 7 on it in mid-october 2013.
I've created a temporary user account on it, which at times I've handle the password (before resetting it) to people just so they could SSH in and type: "uptime".
It is a thing of beauty.
Eight. Years. Of. Uptime.
Re: Tell HN: AWS appears to be down again
#165Earlier quoted context omitted.
Heroku isn’t “low end,” it’s a PaaS built on top of AWS. So you’re really just hearing about another AWS outage lol
They're not saying Heroku is low end. They're saying, "I'm tired of hearing that it's irresponsible to run your own servers." At least, that's what I understood.
Any place I've worked at that managed their own servers (to be fair, the last time I worked at a place like that was 2010) definitely had more protracted downtimes than AWS - it just felt not as bad because we were in control of the situation, but at the end of the day that didn't get us up any faster.
Another side benefit of being with AWS is when you do have an outage, a lot of other people have outages, and so you sort of blend in with the noise. It's not great to be down, but if you're down and also "big service X" who's also an AWS customer is down, it makes your downtime look less like a lack of competence and more like an unavoidable force of nature.
Re: Tell HN: AWS appears to be down again
#166If you haven't seen yet, news is it was a power loss: > 5:01 AM PST We can confirm a loss of power within a single data center within a single Availability Zone (USE1-AZ4) in the US-EAST-1 Region. This is affecting availability and connectivity to EC2 instances that are part of the affected data center within the affected Availability Zone. We are also experiencing elevated RunInstance API error rates for launches wi…
How come they dont have power backups?
Re: Tell HN: AWS appears to be down again
#167Earlier quoted context omitted.
I wonder to what extent this actually becomes less of a problem the more people use AWS. At this point AWS being down just feels like "the internet is down", it's hard for customers to be too mad at any company being down when all their competitors are too. Though I guess there's still probably just lost revenue that could be captured by having better uptime, even if your competitors are down.
This seems like an interesting pendulum swing where the few companies not reliant on AWS could capture significant enough revenue by maintaining uptime during a potential busy season outage.
That depends on them being up the rest of the time. If they have an equal number of outages as AWS, it has the risk to make them look worse (since all attention is on them when they're out).
Re: Tell HN: AWS appears to be down again
#168Earlier quoted context omitted.
I wonder to what extent this actually becomes less of a problem the more people use AWS. At this point AWS being down just feels like "the internet is down", it's hard for customers to be too mad at any company being down when all their competitors are too. Though I guess there's still probably just lost revenue that could be captured by having better uptime, even if your competitors are down.
This seems like an interesting pendulum swing where the few companies not reliant on AWS could capture significant enough revenue by maintaining uptime during a potential busy season outage.
These two outages have been incredibly anomalous, I doubt you'd get much revenue betting that they'll be a common occurrence.
Re: Tell HN: AWS appears to be down again
#169Earlier quoted context omitted.
Your home ISP has 100% uptime? That's incredible.
Lets be real here, we don't need anywhere near 100% ISP uptime to beat AWS over the last couple months...
That depends on what you mean by AWS. I had production workloads in us-east-1 which haven’t been affected by any of these, and others which had only modest degradation. We had control plane issues but the running services were fine.
Put another way: even if your home ISP has had 100% uptime, are you comfortable saying that was true for all of their customers?
Re: Tell HN: AWS appears to be down again
#170I wonder how many 9s AWS is going for. Can't be a lot of 9s anymore.
89.9999 % has a lot of 9s, dare I say military-grade.