Ask HN: Is AWS SSM (or other services) down for anyone else?

#1

Post by dvaun » Fri, Sep 09, 2022, 3:31 PM UTC

We're experiencing production issues due to SSM in us-east-1.

Re: Ask HN: Is AWS SSM (or other services) down for anyone else?

#2

Post by kennu » Fri, Sep 09, 2022, 3:35 PM UTC

I can't deploy on CloudFormation to us-east-1 right now, getting various errors when creating changesets with AWS CDK.

Re: Ask HN: Is AWS SSM (or other services) down for anyone else?

#3

Post by ack210 » Fri, Sep 09, 2022, 3:35 PM UTC

Yes, also in us-east-1 and we currently have an event showing for it in our personal health dashboard (but doesn't seem to be acknowledged on the health dashboard if logged out). It's preventing us from deploying CloudFormation stacks as well right now.

https://imgur.com/a/4fMho2O

Re: Ask HN: Is AWS SSM (or other services) down for anyone else?

#4

Post by kennu » Fri, Sep 09, 2022, 3:48 PM UTC

Amazon Web Services Service Status 18:46

Informational message: Increased API Error Rates and Latency We are investigating increased API error rates and latency in the US-EAST-1 Region. This will impact customers ability to make calls to parameter store.

(from RSS feed, time is EET)

Re: Ask HN: Is AWS SSM (or other services) down for anyone else?

#5

Post by juleska » Fri, Sep 09, 2022, 4:14 PM UTC

Same here, seems to be recovering ...

Re: Ask HN: Is AWS SSM (or other services) down for anyone else?

#6

Post by mmccaff » Fri, Sep 09, 2022, 4:15 PM UTC

kennu wrote: post #2

I can't deploy on CloudFormation to us-east-1 right now, getting various errors when creating changesets with AWS CDK.

same

Re: Ask HN: Is AWS SSM (or other services) down for anyone else?

#7

Post by JimTheDev » Fri, Sep 09, 2022, 4:34 PM UTC

AWS saying it will be an hour until resolution.

Re: Ask HN: Is AWS SSM (or other services) down for anyone else?

#8

Post by colpabar » Fri, Sep 09, 2022, 4:37 PM UTC

Yep. I have seen these errors

botocore.errorfactory.InternalServerError: An error occurred (InternalServerError) when calling the GetParameter operation (reached max retries: 4): Server busy. Please try again.

botocore.errorfactory.InternalServerError: An error occurred (InternalServerError) when calling the GetParameter operation (reached max retries: 4): No Content-Type received. Service returned the HTTP status code: 500

Re: Ask HN: Is AWS SSM (or other services) down for anyone else?

#9

Post by Lapalux » Fri, Sep 09, 2022, 4:38 PM UTC

Getting CloudFront internal errors preventing deployment...

`An error occurred (ServiceUnavailable) when calling the CreateInvalidation operation (reached max retries: 2): CloudFront encountered an internal error. Please try again.`

Re: Ask HN: Is AWS SSM (or other services) down for anyone else?

#10

Post by JimTheDev » Fri, Sep 09, 2022, 5:07 PM UTC

JimTheDev wrote: post #7

AWS saying it will be an hour until resolution.

They just pulled that back. Resolution time is now unknown.