Well, that explains wtf is happening. You guys seeing this on us-east-1 or across?
saw it across us-east-1 mainly
Tell HN: CloudFront Is Having an Outage
11–20 of 25 posts
Re: Tell HN: CloudFront Is Having an Outage
#12AWS Status page doesn't show anything yet: https://health.aws.amazon.com/health/status
Short of something within a few hundred feet of Bezo's desk being actually on fire, AWS's health page never seems to actually report issues.
Re: Tell HN: CloudFront Is Having an Outage
#13AWS Status page doesn't show anything yet: https://health.aws.amazon.com/health/status
Got posted a minute ago.
Operational issue - Amazon CloudFront (Global) Service Amazon CloudFront Severity Informational RSS Elevated Error Rates Jul 18 10:26 AM PDT Between 9:37 AM and 10:13 AM PDT, we experienced elevated error rates for request serviced by the CloudFront Origin Shield and Regional Edge Cache in the US-EAST-1 region. The issue has been resolved and service is operating normally.
Re: Tell HN: CloudFront Is Having an Outage
#14Well, that explains wtf is happening. You guys seeing this on us-east-1 or across?
As the saying goes: Friends don't let friends rely on us-east-1. (It was the very first region, and it shows)
Re: Tell HN: CloudFront Is Having an Outage
#15Well, that explains wtf is happening. You guys seeing this on us-east-1 or across?
As the saying goes: Friends don't let friends rely on us-east-1. (It was the very first region, and it shows)
I will never understand why anyone in our infrastructure team thought for a moment that putting our backup datacenter in us-east-1. And why nobody else tried to get him fired.
That's like buying a backup generator from a guy that wants to meet you in the Walmart parking lot.
Re: Tell HN: CloudFront Is Having an Outage
#16AWS Status page doesn't show anything yet: https://health.aws.amazon.com/health/status
FYI, it takes some pretty high-level approval at AWS for someone to make a change on their "status" page (signaling SLA breaches and money stuff). Not an actual live status page.
Re: Tell HN: CloudFront Is Having an Outage
#17Earlier quoted context omitted.
As the saying goes: Friends don't let friends rely on us-east-1. (It was the very first region, and it shows)
I will never understand why anyone in our infrastructure team thought for a moment that putting our backup datacenter in us-east-1. And why nobody else tried to get him fired. That's like buying a backup generator from a guy that wants to meet you in the Walmart parking lot.
Whoa them fightin words. I bought a generator at a Walmart parking lot and it was fine. I only got stabbed twice.
Generator didn’t actually work though
Re: Tell HN: CloudFront Is Having an Outage
#18AWS Status page doesn't show anything yet: https://health.aws.amazon.com/health/status
Got posted a minute ago. Operational issue - Amazon CloudFront (Global) Service Amazon CloudFront Severity Informational RSS Elevated Error Rates Jul 18 10:26 AM PDT Between 9:37 AM and 10:13 AM PDT, we experienced elevated error rates for request serviced by the CloudFront Origin Shield and Regional Edge Cache in the US-EAST-1 region. The issue has been resolved and service is operating normally.
"elevated error rates" is now my favourite euphemism for a downtime
Re: Tell HN: CloudFront Is Having an Outage
#19AWS Status page doesn't show anything yet: https://health.aws.amazon.com/health/status
Short of something within a few hundred feet of Bezo's desk being actually on fire, AWS's health page never seems to actually report issues.
Bezos is no longer the CEO. It's Andy Jassy.
I agree, AWS (and Amazon Alexa + Ring) seem to be skirting and/or choosing active silence on that "Lambda" outage that took out us-east-1. Alexa had at the very least two full scale outages this year, and Ring was successfully hacked as well.
Amazon's response: "looks like a Lambda... more later" then nothing, nothing, and "it must be a 3rd party skill."
AWS health page: GREEN.
AWS Post-event summaries: non-existent after 2021.
This is quite troubling when you consider what AWS holds in its coffers, along with the number of customers AWS currently has. No transparency, not even strategic transparency, is a definite red flag.
Re: Tell HN: CloudFront Is Having an Outage
#20Well, that explains wtf is happening. You guys seeing this on us-east-1 or across?
As the saying goes: Friends don't let friends rely on us-east-1. (It was the very first region, and it shows)
Having worked at AWS: us-east-1 is probably the easiest region to deploy in (it's old, it's popular, every AWS service is available there), but might just be the worst region to rely on (it's old, it's popular, and every AWS service is available there—but only most of the time).
I vividly remember when AWS Fault Injection Simulator first launched, posting a meme in the #aws-memes slack channel posting a drake meme with the first panel being "Using AWS Fault Injection Simulator", and the second "Deploying in us-east-1".