Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS outage shows internet users 'at mercy' of too few providers, experts say

theguardian.com

Post reply on HN
Print view View original

61–70 of 223 posts

Re: AWS outage shows internet users 'at mercy' of too few providers, experts say

#61

Post by tlogan »

aurumque wrote: post #41
ryandvm wrote: post #36

Man, I did not have "AWS us-east-1 will only have TWO 9s this year" on my bingo card.

For those of us who have been using AWS for almost 20 years now, I can't imagine why anyone would willingly choose us-east-1 for anything. It is the oldest, highest traffic, most critical path region and is subject to turbulence.

I think it is a little complicated. For example, your service might be using full failover but you use API from other service which are down.

Or you might use BART to come to work and you got stuck: https://www.kqed.org/news/12060687/bart-resumes-service-but-...

Re: AWS outage shows internet users 'at mercy' of too few providers, experts say

#62

Post by tayo42 »

bix6 wrote: post #4

Can someone educate me on the solution to this? I assume most organizations, both small and large, just host on whatever provider they know or that costs them the least. If you have budget maybe you deploy to multiple providers for redundancy? But that increases cost and complexity. Who’s going to bother with colo given the cost / complexity? Who’s going to run a server from their office given ISP restrictions and do…

If the cost is worth the complexity then you just do it. Otherwise you don't. How much did a company lose today compared to how much it costs to set it up

And colo and datacenters aren't immune to going down

Re: AWS outage shows internet users 'at mercy' of too few providers, experts say

#63

Post by SoftTalker »

jasode wrote: post #48
impure wrote: post #29

We already have diversification. You can rent a VPS from hundreds of possible companies. And people are very happy with them, it seems every month or two there’s a post here about how some company slashed their cloud bill by switching to a VPS. What we have here is a lock-in and marketing problem.

>You can rent a VPS from hundreds of possible companies. And people are very happy with them, it seems every month or two there’s a post here about how some company slashed their cloud bill by switching to a VPS. Companies are using higher-level "PaaS" suite of services from AWS such as DynamoDB, RedShift, etc and not just the lower-level "IaaS" such as basic EC2 instances or pure containers. Same "lock-in" situation…

> It's going to be a lot more involved

Yes, and you can't escape that by outsourcing it. The complexity is still there, and it will still bite you when your outsourcer fails to manage it.

Re: AWS outage shows internet users 'at mercy' of too few providers, experts say

#64

Post by TYPE_FASTER »

TZubiri wrote: post #50
impure wrote: post #29

We already have diversification. You can rent a VPS from hundreds of possible companies. And people are very happy with them, it seems every month or two there’s a post here about how some company slashed their cloud bill by switching to a VPS. What we have here is a lock-in and marketing problem.

Amazon offers VPS as well, EC2 instances, were those affected? I think they weren't.

> While most operations are recovered, requests to launch new EC2 instances (or services that launch EC2 instances such as ECS) in the US-EAST-1 Region are still experiencing increased error rates.

> We continue to investigate the root cause for the network connectivity issues that are impacting AWS services such as DynamoDB, SQS, and Amazon Connect in the US-EAST-1 Region. We have identified that the issue originated from within the EC2 internal network.

So, kinda? Some global services depend on us-east-1...

> Global services or features that rely on US-EAST-1 endpoints such as IAM updates and DynamoDB Global tables may also be experiencing issues.

Basically, you know it's going to be a bumpy day when us-east-1 has an issue because your ability to run across regions depends on what the issue is what the impact is.

Re: AWS outage shows internet users 'at mercy' of too few providers, experts say

#65

Post by 01HNNWZ0MV43FF »

esafak wrote: post #33
binary132 wrote: post #30

Wow, thanks experts! I never could have figured this out without you :)))

There has to be an Onion article for this.

"No way to prevent this, says only region where this regularly happens"

Re: AWS outage shows internet users 'at mercy' of too few providers, experts say

#66

Post by mads_quist »

TZubiri wrote: post #52
ryandvm wrote: post #36

Man, I did not have "AWS us-east-1 will only have TWO 9s this year" on my bingo card.

Wait, was the whole region affected? Like even if you had an EC2 instance?

No, we run on US East 1 but only EC2. Everything was running smoothly!

Re: AWS outage shows internet users 'at mercy' of too few providers, experts say

#67

Post by candiddevmike »

SoftTalker wrote: post #63
jasode wrote: post #48

Earlier quoted context omitted.

>You can rent a VPS from hundreds of possible companies. And people are very happy with them, it seems every month or two there’s a post here about how some company slashed their cloud bill by switching to a VPS. Companies are using higher-level "PaaS" suite of services from AWS such as DynamoDB, RedShift, etc and not just the lower-level "IaaS" such as basic EC2 instances or pure containers. Same "lock-in" situation…

> It's going to be a lot more involved Yes, and you can't escape that by outsourcing it. The complexity is still there, and it will still bite you when your outsourcer fails to manage it.

Same thing applies to AWS...

Re: AWS outage shows internet users 'at mercy' of too few providers, experts say

#68

Post by labrador »

Kieran Healy @kjhealy@mastodon.social

Always worth taking sentences that use “the Cloud” or “the Internet” and try replacing those phrases with “A shed in Virginia” to see how they hold up. “Our service is fully based in a shed in Virginia”; “All my files are in a shed in Virginia”; “A shed in Virginia was designed to survive a nuclear war”, etc.

https://mastodon.social/@kjhealy/115407725852594322

Re: AWS outage shows internet users 'at mercy' of too few providers, experts say

#70

Post by mads_quist »

mads_quist wrote: post #66
TZubiri wrote: post #52

Earlier quoted context omitted.

Wait, was the whole region affected? Like even if you had an EC2 instance?

No, we run on US East 1 but only EC2. Everything was running smoothly!

Our strategy has always been to use as little higher abstractions from cloud providers as possible. Glad we went this way, saved us quite a bunch of SLA breaches today! I am confident to say that it's "best of both worlds". We get great availability zone redundancy by AWS without having to rely on and pay for all those PaaS stuff the cloud giants offer. Also, we can "fairly easy" migrate to any other cloud provider because we only need Debian instances running.
Post reply on HN