Some sage advice I learned a while ago: "Avoid us-east-1 as much as possible".
But if you use CloudFront, there you go.
AWS us-east-1 outage
591–600 of 1001 posts
Re: AWS us-east-1 outage
#592Me thinks Venmo uses AWS because they are down as well. Status gator has AWS as on off on off on off. I can access my servers hosted in the west coast but I cannot access the AWS console, this is making for an interesting morning.
Re: AWS us-east-1 outage
#593Earlier quoted context omitted.
You and many others here may be conflating two concepts which are actually quite separate. Taking blame is a purely punitive action and solves nothing. Taking responsibility means it's your job to correct the problem. I find that the more "political" the culture in the organization is, the more likely everyone is to search for a scapegoat to protect their own image when a mistake happens. The higher you go up in the…
Every argument I have on the internet is between prescriptive and descriptive language. People tend to believe that if you can describe a problem that means you can prescribe a solution. Often times, the only way to survive is to make it clear that the first thing you are doing is describing the problem. After you do that, and it's clear that's all you are doing, then you follow up with a prescriptive description whe…
[deleted]
Re: AWS us-east-1 outage
#594"some customers may experience a slight elevation in error rates" --> everything is on fire
ECR and API are fucked so it’s impossible to scale anything to the point fire can come out :)
Re: AWS us-east-1 outage
#595Re: AWS us-east-1 outage
#596Earlier quoted context omitted.
Ok, and? I don't doubt it fails in places. That doesn't mean that it doesn't work in practice. Our company does it just fine. We have a high trust, high transparency system and it's wonderful. It's like saying unit tests don't work in practice because bugs got through.
Have you ever considered that the “no-blame” postmortems you are giving credit for everything are just a side effect of living in a high trust, high transparency system? In other words, “no-blame” should be an emergent property of a culture of trust. It’s not something you can prescribe.
Yes, exactly. Culture of trust is the root. Many beneficial patterns emerge when you can have that: more critical PRs, blameless post-mortems, etc.
Re: AWS us-east-1 outage
#597Earlier quoted context omitted.
Yeah, strange, my self-hosted server isn't affected either.
Seems "the cloud" had a major outage less than a month ago, my laptop has a higher uptime. $ 16:04 up 46 days, 7:02, 9 users, load averages: 3.68 3.56 3.18 US East 1 was down just over a year ago https://www.theregister.com/2020/11/25/aws_down/ Meanwhile I moved one of my two internal DNS servers to a second site on 11 Nov 2020, and it's been up since then. One of my monitoring machines has been filling, rotating and…
so does my toaster, oven and microwave. so what? they get used a few times a day, but my production level equipment serves millions in an hour.
Re: AWS us-east-1 outage
#598Earlier quoted context omitted.
I worked at Walmart Technology. I bravely wrote post mortem documents owning the fault of my team (100+ people), owning both technically and also culturally as their leader. I put together a plan to fix it and executed it. Thought that was the right thing to do. This happend two times in my 10 year career there. Both times I was called out as a failure in my performance eval. Second time, I resigned and told them to…
That's shockingly stupid. I also worked for a major Walmart IT services vendor in another life, and we always had to be careful about how we handled them, because they didn't always show a lot of respect for vendors. On another note, thanks for building some awesome stuff -- walmart.com is awesome. I have both Prime and whatever-they're-currently-calling Walmart's version and I love that Walmart doesn't appear to mix…
walmart.com user design sucks. My particular grudge right now is - I'm shopping to go pickup some stuff (and indicate "in store pickup) and each time I search for the next item, it resets that filter making me click on that filter for each item on my list.
Re: AWS us-east-1 outage
#599Earlier quoted context omitted.
They are still lying about it, the issues are not only affecting the console but also AWS operations such as S3 puts. S3 still shows green.
It's certainly affecting a wider range of stuff from what I've seen. I'm personally having issues with API Gateway, CloudFormation, S3, and SQS
Our corporate ForgeRock 2FA service is apparently broken. My services are behind distributed x509 certs so no problems there.
Re: AWS us-east-1 outage
#600Amazon is having an outage in us-east-1, and it is bleeding over elsewhere, like eu: https://status.aws.amazon.com/
Those 2 services that are being marked as having problems in other regions have fairly hard dependency on us-east-1. So that would be why.