Live data from Hacker News

AWS S3 Outage

news.ycombinator.com

61–70 of 142 posts

Re: AWS S3 Outage

#61
post #49

Earlier quoted context omitted.

Mapbox and Hipchat too https://www.mapbox.com/status/ https://status.hipchat.com/

Feels like internet is not anymore distributed thing that if one website/node goes down others keep working... Feels like in future... If cloud provider goes down... All internet will stop working :)

This is pretty much the case. Years of evangelising the idea that (a) everybody should be on Amazon and (b) everybody should be on the cheapest regions of Amazon mean that while the underlying datacentres are probably much better managed, individually speaking, than the tapestry of colos that made up the world a decade ago, an outage has much more wide-ranging effects than you'd get at that point.

Re: AWS S3 Outage

#62
post #55

Latest Update from http://status.aws.amazon.com/ : 1:52 AM PDT We are actively working on the recovery process, focusing on multiple steps in parallel. While we are in recovery, customers will continue to see elevated error rate and latencies.

The icon got yellow. I repeat: YELLOW

Re: AWS S3 Outage

#63
post #59

even ec2 is down: Amazon Elastic Compute Cloud (N. Virginia) Increased API Error Rates 12:51 AM PDT We are investigating increased error rates for the EC2 APIs and launch failures for new EC2 instances in the US-EAST-1 Region.

EC2 is not down.

It's up, but launching new instances doesn't work due to the S3 issue.

Re: AWS S3 Outage

#64
post #55

Latest Update from http://status.aws.amazon.com/ : 1:52 AM PDT We are actively working on the recovery process, focusing on multiple steps in parallel. While we are in recovery, customers will continue to see elevated error rate and latencies.

The icon got yellow. I repeat: YELLOW

my service looks pretty red now

Re: AWS S3 Outage

#65
post #55

Latest Update from http://status.aws.amazon.com/ : 1:52 AM PDT We are actively working on the recovery process, focusing on multiple steps in parallel. While we are in recovery, customers will continue to see elevated error rate and latencies.

The EC2 launch thing is still green even though all launches fail for us.

Re: AWS S3 Outage

#66
post #55

Latest Update from http://status.aws.amazon.com/ : 1:52 AM PDT We are actively working on the recovery process, focusing on multiple steps in parallel. While we are in recovery, customers will continue to see elevated error rate and latencies.

The icon got yellow. I repeat: YELLOW

It only goes red when a nuclear event occurs, obliterating most of humanity and only the machines remain.

Re: AWS S3 Outage

#67
It's two hours. Luckily it happenned around 12:00AM otherwise we are having a bad day.

Whomever being a DevOps or SysAdmin probably cannot sleep tonight :(.

In our case, we put Fastly on top of our assets/images so that only a partil of request get errors. The cached object on Fastly is still fine.

Re: AWS S3 Outage

#68

It's two hours. Luckily it happenned around 12:00AM otherwise we are having a bad day. Whomever being a DevOps or SysAdmin probably cannot sleep tonight :(. In our case, we put Fastly on top of our assets/images so that only a partil of request get errors. The cached object on Fastly is still fine.

Actually on some parts of the world it's not 12:00AM you know, and there are sysadmins/devops there too :-)

Re: AWS S3 Outage

#69

It's two hours. Luckily it happenned around 12:00AM otherwise we are having a bad day. Whomever being a DevOps or SysAdmin probably cannot sleep tonight :(. In our case, we put Fastly on top of our assets/images so that only a partil of request get errors. The cached object on Fastly is still fine.

haha if two hours is nothing to you, that's pretty lucky. We work 24x7 (global customers, yes US is a bit busier but not by a lot) and we're an Australian company, so not a whole lot of fun.
Post reply on HN