Live data from Hacker News

AWS EC2 Having Issues

status.aws.amazon.com

21–30 of 52 posts

Re: AWS EC2 Having Issues

#21

interesting they use blue as the color that indicate an "issue". at a glance, looks like nothing to see here. i have a feeling that if the entire AWS infrastructure were to go offline somehow they would refer to that as 'increase error rates'

I suspect it's an accessibility decision.

Blue is more distinguishable from green by colour-blind folks. Red is not distinguishable.

Re: AWS EC2 Having Issues

#23

interesting they use blue as the color that indicate an "issue". at a glance, looks like nothing to see here. i have a feeling that if the entire AWS infrastructure were to go offline somehow they would refer to that as 'increase error rates'

Downtime is kinda hard to grok for huge distributed services like this.

If you’re down when anyone gets errors you’ll probably always be down — not really useful.

If you’re down when everyone gets errors you’ll probably always be up — again not really useful.

From a customer perspective an increase in errors means nothing if it doesn’t affect you or everything if it does so even a flat percentage is likely not all that useful except as a probability that any issues you’re seeing are your fault.

Re: AWS EC2 Having Issues

#25

interesting they use blue as the color that indicate an "issue". at a glance, looks like nothing to see here. i have a feeling that if the entire AWS infrastructure were to go offline somehow they would refer to that as 'increase error rates'

The first color I noticed on the page was the orange RSS icons, although that's probably because I was primed by the submission's headline.

Re: AWS EC2 Having Issues

#26
post #23

interesting they use blue as the color that indicate an "issue". at a glance, looks like nothing to see here. i have a feeling that if the entire AWS infrastructure were to go offline somehow they would refer to that as 'increase error rates'

Downtime is kinda hard to grok for huge distributed services like this. If you’re down when anyone gets errors you’ll probably always be down — not really useful. If you’re down when everyone gets errors you’ll probably always be up — again not really useful. From a customer perspective an increase in errors means nothing if it doesn’t affect you or everything if it does so even a flat percentage is likely not all th…

On this topic, a popular previous post on HN is 'Your nines are not my nines' [1] by rachelbythebay.

[1] https://news.ycombinator.com/item?id=20451714

Re: AWS EC2 Having Issues

#27
GCP also experienced problems. A really nice one were if you deleted a service account you end up with restart loop:

> We are experiencing an issue with Google Kubernetes Engine. Removing Service Account from GKE might lead to infinite cluster master restarts. Please refrain from removing GKE service accounts.

Re: AWS EC2 Having Issues

#28

Never run prod in us-east-1.

I remember being bitten by an RDS update, running in ap-southeast-2.

Took out a few of our production databases, seems like Australia is the beta testing region for at least a few services.

At least in the US you have the choice of other regions.

Re: AWS EC2 Having Issues

#29
post #7

Earlier quoted context omitted.

That joke is getting pretty tired by now and restating it seems unlikely to be helpful for anyone who's affected by the rate-limited API calls in question.

It's not a joke. The us-east-1 region is one of if not the oldest. Some of the infrastructure is aging because it's hard to phase out. Amazon has learned a lot since then and the newer regions have reaped all the benefits, where Virginia hasn't. There are components that exist in that region and don't exist in others but as a rule, I use those as remote resources and keep all my critical infra in Ohio, if I need to b…

Have you actually measured it? I’ve had production workloads in us-east-1 for over a decade and haven’t seen it be notably different than the other regions. There’ve been a handful of times where there’s been a truly regional outage (17 minutes of network connectivity in 2011) and a couple of broader service outages (S3, IAM), but it’s not like the other regions never have problems.

In general most of the time when I’ve heard people say “us-east-1 is unreliable” it’s ended up that they’re really telling you that they were in just one AZ, thought the RDS multiple AZ mode was too expensive, etc. and standard architectural guidance would have taken care of it.

Post reply on HN