Now is probably a good time to plug some of the open source alternatives to vendor locked in identity solutions: - https://github.com/ory - https://github.com/dexidp/dex - https://github.com/authelia/authelia - https://github.com/keycloak/keycloak - https://www.gluu.org/ - https://github.com/accounts-js/accounts
AWS Cognito is having issues and health dashboards are still green
151–160 of 369 posts
Re: AWS Cognito is having issues and health dashboards are still green
#152Earlier quoted context omitted.
>Part of the problem is, engineers love shiny things. Status pages are fundamentally boring things. Who wants to work on them? I would work on a status page. It's a interesting problem, creating tests that prove services are viable at a place like AWS would be fun. However what I don't want to deal with is some director of so and so I never heard of yelling at me at 3 in the morning because my status page reported th…
Congratulations, you're already complicating the status page. The status page shouldn't be figuring out what the status of any service is. It's impossible to do without a lot of contextual information about a service and understanding how to evaluate service impact, something that is continually in flux. It just needs to be a page that is updated manually. AWS has a 24x7 incident management team that could / should d…
Updated manually by whom?
I'm afraid you're shifting the complexity to a manual process.
I agree that it doesn't have to, and perhaps should not, be fully automated. But automating some parts will help not waste time on last minute arguments.
Re: AWS Cognito is having issues and health dashboards are still green
#153Earlier quoted context omitted.
It is kind of perplexing that AWS dogfoods its own status page. I remember during the massive S3 outage a few years ago that their status page remained green almost the entire time because the red/green/blue icons for the status was stored in... wait for it... S3. You'd think they would have learned from that.
> It is kind of perplexing that AWS dogfoods its own status page. > You'd think they would have learned from that. They did. The page has been updated numerous times since the start of this incident.
From the status page:
> This issue has also affected our ability to post updates to the Service Health Dashboard.
Just seems so ridiculous that they have trouble reporting the impaired status of their system due to... the impaired status of that same system.
Re: AWS Cognito is having issues and health dashboards are still green
#154Earlier quoted context omitted.
Eh? Brokering amongst multiple trusts (and managing protocol transition) is almost the raison d'etre for lifting token issuance out of your app and into ADFS, Okta, Auth0, etc. Of course you'll have to deal with home realm discovery--really need to go in with open eyes on that one.
Yes, but cognito endpoints and pools ids are regional and globally unique, and there is no way that I know of to setup duplicate userpools in multiple regions and have requests served by either region. That means the customer IDP side would need to have two different SAML apps configured for each region...
Ah, I see what you mean. It does seem like you'd want a more complex arrangement of trusts to keep things simple on the leaves; or else avoid using a product that requires generating a hundred scattered security authorities.
Re: AWS Cognito is having issues and health dashboards are still green
#155I think we are learning everything that uses AWS Kinesis internally which is cool. It’s always fascinating to learn how AWS works on the backend.
I work at AWS. I can tell you surely enough it's not pretty or easy to work with. Design and architecture are great here but implementation of that is pretty crap...
Why use it then? (api is crap, uptime is crap, limits are crap... politics?)
Re: AWS Cognito is having issues and health dashboards are still green
#156Earlier quoted context omitted.
Updating the status dashboard is pretty low priority for operators trying to resolve this issue. It requires escalation up the management chain and careful wording.
By design. If it was a good faith attempt to report status, it would be automatically updated from a flock of canaries instead of through a slow, political process.
Even that would be meaningless at the scale of AWS.
"A top of rack switch let out the blue smoke and it'll be ~30 before we can re-rack it" would impact what fraction of a fraction of a percent of canaries? Irrelevant to me, unless of course my VM lives on a box backed by that switch. ;)
The status dashboard exists for us to laugh at when things break and to convince C*Os that everything is fine. That's it.
Re: AWS Cognito is having issues and health dashboards are still green
#157Earlier quoted context omitted.
> Can anyone explain why status pages are so difficult. What is an outage? When does an outage reach sufficient scale that updating the status page is the right thing to do? I used to work for AWS, and now work for another cloud provider. One thing that's hard to communicate is the sheer scale that these services operate at, what that means architecturally, and how they tend to break. Outages, even just slight degrad…
Posting percentages instead of green/red would fix all of these, no?
Not really. People will automatically assume they were in that impacted percentage and that what was happening with their stuff was entirely AWS's fault.
Re: AWS Cognito is having issues and health dashboards are still green
#158Isn't it common practice to host your status board on someone else's infrastructure? In 2017 there was an S3 issue that supposedly affected their ability to post. I believe they said that they were updating how they posted to the status board so that there would no longer be a dependency on S3. Well, I guess whatever they're dependent on now broke.
That day was a nightmare for a lot of people - it wasn't just S3 that went down, it was like all of US-EAST. Luckily my company decided against multi-az for the cost savings so I spent all day firefighting.
Multi-AZ doesn't help when a whole region is down, unless you're referring to multi-region AZs (e.g us-east-1a and us-west-1a)
Re: AWS Cognito is having issues and health dashboards are still green
#159Earlier quoted context omitted.
Updating the status dashboard is pretty low priority for operators trying to resolve this issue. It requires escalation up the management chain and careful wording.
>for operators trying to resolve this issue It's a shame Amazon doesn't have thousands of employees to divide these tasks between different people, as it is only these busy operators who could update this status page. If you're right, why have the status page then? It is useless by your definition yes?
Not to mention it doesn't take a technical person to update the status page.
Its even more frustrating when you are aware of problems early on and start talking to support and THEY don't even know about problems yet.
Maybe the thousands of people is what prevents status from being updated, everyone tries to hide their own faults internally even
Re: AWS Cognito is having issues and health dashboards are still green
#160Earlier quoted context omitted.
Updating the status dashboard is pretty low priority for operators trying to resolve this issue. It requires escalation up the management chain and careful wording.
>for operators trying to resolve this issue It's a shame Amazon doesn't have thousands of employees to divide these tasks between different people, as it is only these busy operators who could update this status page. If you're right, why have the status page then? It is useless by your definition yes?
Just because it has a lag from “issues reported” to “confirmed outage” doesn’t mean it’s useless. Non-green means there are issues and Amazon is aware of them.