Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS Cognito is having issues and health dashboards are still green

status.aws.amazon.com

Post reply on HN
Print view View original

221–230 of 369 posts

Re: AWS Cognito is having issues and health dashboards are still green

#221

Post by necubi »

outworlder wrote: post #179
PragmaticPulp wrote: post #172

We hired an engineer out of Amazon AWS at a previous company. Whenever one of our cloud services went down, he would go to great lengths to not update our status dashboard. When we finally forced him to update the status page, he would only change it to yellow and write vague updates about how service might be degraded for some customers. He flat out refused to ever admit that the cloud services were down. After some…

No idea what happens on AWS as I don't work there, but I have another perspective on this. There are perverse incentives to NOT update your status dashboard. Once I was asked by management to _take our status dashboard down_ . That sounded backwards, so I dug a bit more. Turns out our competitor was using our status dashboard as ammo against us in their sales pitch. Their claim was that we had too many issues and wer…

Yeah, had the same experience at a previous company. It's very frustrating that your transparency gets used against you by unscrupulous competitors.

Re: AWS Cognito is having issues and health dashboards are still green

#222

Post by kdmytro »

MadVikingGod wrote: post #211
stevepotter wrote: post #193

Earlier quoted context omitted.

I work at AWS now and can second that. Nobody is happy when things break, but COEs are looked a positively and are circulated constantly to prevent repeats.

Not at AWS, retail Amazon, but what I saw was COEs were either normal business process or PIP material depending on which org you worked for. And sometimes just the excuse to get you gone. Where I was about 99.9% of the COEs where just a lesson learned and new process to prevent it. There was one that was basically used as a tool to remove a VERY good engineer, that didn't mesh well with new leadership. A sister org,…

What is PIP?

Re: AWS Cognito is having issues and health dashboards are still green

#223

Post by sk5t »

myleshenderson wrote: post #190
drfritznunkie wrote: post #122

Earlier quoted context omitted.

Yes, but cognito endpoints and pools ids are regional and globally unique, and there is no way that I know of to setup duplicate userpools in multiple regions and have requests served by either region. That means the customer IDP side would need to have two different SAML apps configured for each region...

This was shared with me today: https://medium.com/@nealrp/aws-cross-region-cognito-replicat...

Quite superficial don’t you think?

Re: AWS Cognito is having issues and health dashboards are still green

#225

Post by whoknew1122 »

kdmytro wrote: post #222
MadVikingGod wrote: post #211

Earlier quoted context omitted.

Not at AWS, retail Amazon, but what I saw was COEs were either normal business process or PIP material depending on which org you worked for. And sometimes just the excuse to get you gone. Where I was about 99.9% of the COEs where just a lesson learned and new process to prevent it. There was one that was basically used as a tool to remove a VERY good engineer, that didn't mesh well with new leadership. A sister org,…

What is PIP?

Performance Improvement Plan. Basically 'this is what you need to improve if you want to keep working here.'

Lots of people bad at their jobs blame the PIP system for their failure at Amazon.

Re: AWS Cognito is having issues and health dashboards are still green

#226

Post by parthdesai »

kdmytro wrote: post #222
MadVikingGod wrote: post #211

Earlier quoted context omitted.

Not at AWS, retail Amazon, but what I saw was COEs were either normal business process or PIP material depending on which org you worked for. And sometimes just the excuse to get you gone. Where I was about 99.9% of the COEs where just a lesson learned and new process to prevent it. There was one that was basically used as a tool to remove a VERY good engineer, that didn't mesh well with new leadership. A sister org,…

What is PIP?

Most likely, Performance Improvement Plan

Re: AWS Cognito is having issues and health dashboards are still green

#227

Post by smithza »

MadVikingGod wrote: post #211
stevepotter wrote: post #193

Earlier quoted context omitted.

I work at AWS now and can second that. Nobody is happy when things break, but COEs are looked a positively and are circulated constantly to prevent repeats.

Not at AWS, retail Amazon, but what I saw was COEs were either normal business process or PIP material depending on which org you worked for. And sometimes just the excuse to get you gone. Where I was about 99.9% of the COEs where just a lesson learned and new process to prevent it. There was one that was basically used as a tool to remove a VERY good engineer, that didn't mesh well with new leadership. A sister org,…

PIP : Performance Improvement Plan

I assume this to mean that it is an element of an individual's PIP, a formal process to set guides for getting someone to commit to a higher level of achievement.

Re: AWS Cognito is having issues and health dashboards are still green

#228

Post by ben509 »

Bombthecat wrote: post #219
driverdan wrote: post #210

Five hours later and nothing has changed. For a company like Amazon this should be unacceptable. Before someone replies and says use a different AZ, that's not possible for everyone. If you use a 3rd party service that is hosted on us-east-1 you can't do anything about it. For example, many Heroku services are broken because of this.

I think the deeper problem is the interconnectivity between services and their apis. It's too complicated to maintain...

Yeah, the cascading failure of all the other services is a deep architectural issue.

Having lots of services that do one thing and one thing well makes a lot of sense. Breaking them out into separate components brings a level of visibility into the system. And it's AWS's whole business model.

But it does mean that, fundamentally, service X is available when and only when (WAOW?) services A, B, C, etc. are all available. Its uptime is no greater than min(uptime(A), uptime(B), etc)

I'm trying to rework the authentication for our application and integrate it with our parent company's systems. As we talk to other teams, I see all these architecture diagrams where the solution to every problem is Yet Another Service, to where you're running a real rube goldberg machine.

Re: AWS Cognito is having issues and health dashboards are still green

#229

Post by LegitShady »

smithza wrote: post #227
MadVikingGod wrote: post #211

Earlier quoted context omitted.

Not at AWS, retail Amazon, but what I saw was COEs were either normal business process or PIP material depending on which org you worked for. And sometimes just the excuse to get you gone. Where I was about 99.9% of the COEs where just a lesson learned and new process to prevent it. There was one that was basically used as a tool to remove a VERY good engineer, that didn't mesh well with new leadership. A sister org,…

PIP : Performance Improvement Plan I assume this to mean that it is an element of an individual's PIP, a formal process to set guides for getting someone to commit to a higher level of achievement.

While on its face PIP is a guide to getting someone to commit to a higher level of improvement, for many companies its a formal warning that you need to shape up or you're going to be let go.
Post reply on HN