Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS US East is experiencing high error rates on several services

news.ycombinator.com

Post reply on HN
Print view View original

171–180 of 184 posts

Re: AWS US East is experiencing high error rates on several services

#171

Post by antod »

toomuchtodo wrote: post #166
yeukhon wrote: post #165

Earlier quoted context omitted.

Nope. I still disagree. No service can guarantee 99.999999% unless you discount software upgrade. You just cannot. If you think those nines include software upgrades, you are probably over optimistic.

> No service can guarantee 99.999999% Don't advertise it if you can't offer it then. > If you think those nines include software upgrades, you are probably over optimistic. If you advertise a product with a specific SLA, and you can't meet that SLA, you're a liar. Don't try to blame the victim because of inaccurate/untruthful marketing or engineering.

SLAs are just contractual thresholds for getting some specified redress if not met. They are not promises.

Not meeting a SLA is not lying.

Re: AWS US East is experiencing high error rates on several services

#172

Post by rdtsc »

zkhalique wrote: post #145

This is why 99.99999999% uptime is a fallacy It is not really measuring the time you're going to be up. That interpretation is based on faulty assumptions. It's like the statement "the sun will burn out before one bit is flipped" is wrong. It is quite likely that by that time, all the bits will be gone. https://signalvnoise.com/posts/3067-lets-get-honest-about-up...

Yeah it is bullshit. They just had a failure so now they can claim, oh it is still 9 9s it is just that it is over 400 billion years averaged, not like you assumed, 10 billions. So legally still cool though...

Re: AWS US East is experiencing high error rates on several services

#173

Post by rdtsc »

cle wrote: post #120
nasalgoat wrote: post #8

This seems like another reason to not rely on Amazon-specific services, other than the obvious vendor lock-in. At least in the event of an instance outage you could conceivably migrate off Amazon to another VPS provider. No one using DynamoDB has an alternative.

How would this have helped? In the event of an AWS outage, you have all-hands of one of the biggest tech companies in the world working to fix your infrastructure. This outage lasted a few hours, wouldn't it take you that long to switch providers anyway? If you don't abdicate responsibility for maintaining the infrastructure, you lose a lot of the operational benefits of using a cloud service...in general, I think it…

> wouldn't it take you that long to switch providers anyway?

It might, or might not, depends how you built it.

> you have all-hands of one of the biggest tech companies in the world working to fix your infrastructure.

And it still was out for hours... You might see it as a great success, but you can also interpret it as "what good is it, even if with so many people behind the scenes they still failed for hours...".

The problem is that because that services ends up being embedded in so many services and products, half the internet ends being down. Even Amazon dogfoods their own stuff so services rely on each other. Dynamo is down, maybe SQS will be down or analytics as well.

Re: AWS US East is experiencing high error rates on several services

#174

Post by toomuchtodo »

antod wrote: post #171
toomuchtodo wrote: post #166

Earlier quoted context omitted.

> No service can guarantee 99.999999% Don't advertise it if you can't offer it then. > If you think those nines include software upgrades, you are probably over optimistic. If you advertise a product with a specific SLA, and you can't meet that SLA, you're a liar. Don't try to blame the victim because of inaccurate/untruthful marketing or engineering.

SLAs are just contractual thresholds for getting some specified redress if not met. They are not promises. Not meeting a SLA is not lying.

I used SLA to communicate an advertised/marketed level of service. In this case, I agree, that SLA is the wrong term as there is no contractual agreement.

Re: AWS US East is experiencing high error rates on several services

#175

Post by chaosmonkey »

janson0 wrote: post #12

SQS is the specific service giving me a ton of trouble right now. Hope they resolve this quickly. Had rayguns about sqs all night heh. So are they saying they are throttling SQS because of the DynamoDB issue?

I have seen multi-hour SQS outages recently. I'm thinking of options for how we can go about preventing an application failure if this happens again.

* If adding to SQS fails, temporarily store the item on disk or S3, then add to SQS when it's back up?

* any other options?

Re: AWS US East is experiencing high error rates on several services

#178

Post by robotfelix »

watson wrote: post #138
jgwmaxwell wrote: post #105

Earlier quoted context omitted.

Interesting that the outage in us-east-1 is causing huge issues for Heroku in eu-west-1 too - obviously a region for Heroku isn't self sufficient.

Where do you see that it's causing issues for Heroku in eu-west-1? I have a lot of dynos in that region and I don't see any issues

Usually Heroku mentions a specific region in any status updates that don't affect all regions. For example, a status update 11 days ago stated "We're currently seeing increased routing latency in the US region. Investigation is underway."

AFAIK this outage only affected dyno restarting (which may have been triggered by a number of reasons) and creation of new dynos. Perhaps your EU dynos were lucky enough to not have done either of these things during the outage?

Re: AWS US East is experiencing high error rates on several services

#179

Post by Tiksi »

clebio wrote: post #176
smcnally wrote: post #107

Earlier quoted context omitted.

There is only Zuul?

Can you hint at what this means, for the uninitiated?

I think he was making a reference to ghostbusters: http://knowyourmeme.com/memes/there-is-no-dana-only-zuul

Re: AWS US East is experiencing high error rates on several services

#180

Post by anthony_franco »

cbhl wrote: post #126
TheAceOfHearts wrote: post #124

Earlier quoted context omitted.

Why move it if it works fine as-is? I think Waze was bought by Google in 2013 and AFAIK, they're also still in AWS.

The PR is still awkward: "come use Google Cloud Services!" while it's not good/easy enough for your acquisitions to migrate to it from AWS.

Even Amazon.com isn't fully reliant on AWS. They're not using Route 53 for their DNS (or even for their backup DNS).
Post reply on HN