AWS Cognito is having issues and health dashboards are still green • Hacker News Board

Re: AWS Cognito is having issues and health dashboards are still green

#11

Post by snazz » Wed, Nov 25, 2020, 3:41 PM UTC

_0o6v wrote: post #8

> It's not posted on SHD as the issue has impacted our ability to post there. Is that not a massive catch-22 for a service dashboard?

This has happened a few times before, actually. Dogfooding is good, but not for status pages!

Cloudflare does it right for their status page (https://www.cloudflarestatus.com). They don't use Cloudflare itself for it (you can tell because /cdn-cgi/trace returns nothing), the actual backend is Atlassian Statuspage, their TLS certificate is issued by Let's Encrypt instead of Cloudflare itself, and it's on a completely separate domain for DNS purposes.

Re: AWS Cognito is having issues and health dashboards are still green

#12

Post by LeoTM » Wed, Nov 25, 2020, 3:42 PM UTC

Experiencing 504's from Cognito too, our users can't log in.

"amazon-cognito-identity-js": "^3.2.2" "aws-amplify": "^2.2.2"

Re: AWS Cognito is having issues and health dashboards are still green

#13

Post by that_guy_iain » Wed, Nov 25, 2020, 3:43 PM UTC

simlevesque wrote: post #4

I'm getting tired of that bullshit. Just admit it.

AWS' conditions for what healthy is and everyone elses is completely different. Kinda makes me wonder what their internals are like.

Re: AWS Cognito is having issues and health dashboards are still green

#14

Post by rexreed » Wed, Nov 25, 2020, 3:43 PM UTC

mushufasa wrote: post #5

what a scam. who can hold them accountable for cheating those who paid for uptime guarantees? I guess the lawyers of those who paid for uptime guarantees...

This is a relevant comment. I agree. Who will hold them accountable for violating uptime guarantees? Nobody? Then what's the point or purpose of an uptime guarantee? Marketing value?

Re: AWS Cognito is having issues and health dashboards are still green

#15

Post by tysoncadenhead » Wed, Nov 25, 2020, 3:44 PM UTC

Maybe AWS should put their dashboards on GCP

Re: AWS Cognito is having issues and health dashboards are still green

#16

Post by dsagal » Wed, Nov 25, 2020, 3:45 PM UTC

Banner on top of https://status.aws.amazon.com/ just has an update from 8:36AM PST -- just removed -- even thought it's only 7:42AM PST. I guess it's really manual firefighting there.

Re: AWS Cognito is having issues and health dashboards are still green

#17

Post by dtjones » Wed, Nov 25, 2020, 3:48 PM UTC

We're getting 504 for our well-known jwks file

And request timeouts against cognito-idp.us-east-1.amazonaws.com

And the cognito console won't load

Re: AWS Cognito is having issues and health dashboards are still green

#18

Post by turdnagel » Wed, Nov 25, 2020, 3:48 PM UTC

Isn't it common practice to host your status board on someone else's infrastructure?

In 2017 there was an S3 issue that supposedly affected their ability to post. I believe they said that they were updating how they posted to the status board so that there would no longer be a dependency on S3. Well, I guess whatever they're dependent on now broke.

Re: AWS Cognito is having issues and health dashboards are still green

#19

Post by gt565k » Wed, Nov 25, 2020, 3:49 PM UTC

mushufasa wrote: post #5

what a scam. who can hold them accountable for cheating those who paid for uptime guarantees? I guess the lawyers of those who paid for uptime guarantees...

wah wah wah, my boss is chewing my ear out cuz I can't explain to him that it's a vendor (AWS) outage

host on-premise then, and deal with power outages, and infrastructure management, and failing hardware

shit happens, drink a coffee and sit back for 30 mins

Re: AWS Cognito is having issues and health dashboards are still green

#20

Post by arusahni » Wed, Nov 25, 2020, 3:52 PM UTC

All my CloudWatch alerts are firing "OK" transitions, and AWS ES isn't displaying any known instances