Live data from Hacker News

Is Amazon's cloud service too big to fail?

fnlondon.com

91–100 of 164 posts

Re: Is Amazon's cloud service too big to fail?

#91
post #18

Earlier quoted context omitted.

I really hate the "too big to fail" meme and I strongly agree with Bernie in that if you are too big to fail you are too big to exist. That should be the priority.

Agreed, it seems like the software approach as well. You wouldn't want a class or a piece of code to be "too big" to fail, you'd refactor it into smaller pieces which can be overviewed more easily.

It's really a basic engineering principle. Smaller and more distributed systems are more reliable.

Re: Is Amazon's cloud service too big to fail?

#92

Hasn't anyone heard of disaster recover plans? I used to work at a medium sized insurance company and every year we had a project to update our disaster recovery plans. Including our main inhouse datacenter going down. If it was a critical system you'd better have a plan to get it back up in like 4 hours. and those were business critical we didn't have any life critical systems.

What's the disaster plan for "DynamoDB doesn't exist any more"? There is literally nothing else like it in the world. I don't know of an idiot proof queue system that can handle the scales SQS can take either.

Re: Is Amazon's cloud service too big to fail?

#93
post #3

Even at a smaller scale it is a little nerve-wracking to know be so reliant on one provider. If AWS tanks there's a fair amount of code that'd need to be changed just to switch over to Azure or GCE. Failover with, e.g., email providers is easy enough, but the entire cloud stack (for lack of better terms) is a completely different ballgame.

I warn other developers at my company about this. When new projects spin up they're often very excited about using new Amazon services and will make any excuse to choose an AWS product over a stable open source solution. If I were a manager, I'd be very worried over the vendor lock-in.

I don't understand the preference for AWS over open source in many cases. Their services are "reliable", but they often have minute restrictions that will eventually bite you. You also end up having to pay for something you could get for free. Why use SNS/SQS when there are free pubsub/message buses out there? Most of the other devs justify this with the argument of not having to maintain the software themselves. "But RabbitMQ might crash! We don't have to worry about that with AWS!"

Anyway, I typically minimize the AWS services I use (S3, EC2, ECS) so I don't dread the day AWS blows up or, more likely, some VP or exec says we're moving to GCP/Azure because we got a better deal.

Re: Is Amazon's cloud service too big to fail?

#94
post #43

Earlier quoted context omitted.

I feel like the point of the article we're commenting on is that AWS is far from "a few websites"

You can live without Netflix much longer than you can without a flushing toilet, is the point I'm making. Yet we don't have armed guards patrolling the sewageworks... It's a matter of priority how finite security personnel are deployed.

If AWS went down hard, it would be more than just "I can't watch Netflix".

Some people wouldn't be able to do their computer work, send receive emails, others might not receive their paychecks or be able to pay bills.

Re: Is Amazon's cloud service too big to fail?

#95

Earlier quoted context omitted.

I feel like the point of the article we're commenting on is that AWS is far from "a few websites"

Knocking out all of AWS is very different from knocking out a single data center.

Especially because AWS regions are broken up into multiple availability zones (data centres in the same area). So taking out a single data centre won't do much if the AWS customers have correctly designed their systems for high-availability (ie having redundant instances in other AZs/regions with their data backed up elsewhere).

Re: Is Amazon's cloud service too big to fail?

#96
post #70

Earlier quoted context omitted.

There are some open source implementations of parts of the APIs of cloud providers that might help someone a bit when trying to migrate. For example, Minio [1] [2] implements the AWS S3 v4 API. [1]: https://news.ycombinator.com/item?id=12392081 [2]: https://minio.io/

We[1] have seen a fair number of requests for managed services as devs claim that "we don't have the time or skills to maintain these open source components" (quoting verbatim from a request on Intercom). I don't think this is about having open source substitutes in all the cases. Personally not a fan of how/where the build vs. buy debate is playing out here. [1]: https://hasura.io

"we don't have the time or skills to maintain these open source components" translates to "we don't want to install a monitoring solution that restarts a service when it fails, or think about design in regard to component failure".

It's such a poor argument. I was a developer long before AWS appeared and I've used so many open sources packages that were profoundly reliable. In many cases it just takes a daemon restart. And while it's not exciting to set up some of that stuff, it's far more tolerable than writing a CloudFormation template.

Re: Is Amazon's cloud service too big to fail?

#97

Earlier quoted context omitted.

Knocking out all of AWS is very different from knocking out a single data center.

Especially because AWS regions are broken up into multiple availability zones (data centres in the same area). So taking out a single data centre won't do much if the AWS customers have correctly designed their systems for high-availability (ie having redundant instances in other AZs/regions with their data backed up elsewhere).

A single availability zone can be spread across many physical data centres.

According to this rackspace article, the largest AZ has 5 data centres.

https://blog.rackspace.com/aws-101-regions-availability-zone...

Re: Is Amazon's cloud service too big to fail?

#98
post #46

Earlier quoted context omitted.

I'd be very surprised if big users of AWS or Azure pay the rates advertised on the public web sites.

Like most large business, AWS has a sales department and sales engineers that behave like you expect. Rates are absolutely negotiable.

Our AWS rep is nice and cheery. He'll come into our office twice a year and bring sales engineers to hear about our upcoming projects. There's one lead developer on our team who keeps imagining systems that use half a dozen AWS services for "big data". The AWS dudes always end up talking to him the most and they definitely bait him with various pitches and, of course, feed his ego. Good thing that he's so disorganized and delayed that he never has a chance to waste company money on all that bullshit.

Re: Is Amazon's cloud service too big to fail?

#99

Earlier quoted context omitted.

Knocking out all of AWS is very different from knocking out a single data center.

Especially because AWS regions are broken up into multiple availability zones (data centres in the same area). So taking out a single data centre won't do much if the AWS customers have correctly designed their systems for high-availability (ie having redundant instances in other AZs/regions with their data backed up elsewhere).

That's a big if. The S3 outage in February this year showed that there are a lot of sites that aren't designed for this.

Re: Is Amazon's cloud service too big to fail?

#100

If your architecture means your system goes down if AWS is down, then the question becomes can you replace AWS with something better that you can build, have means to build, have time to build, can keep running, can get enough momentum in term of sheer size of customer base to fund the upkeep of the platform? If you can't build/run a better AWS replacement then it's a mute point, isn't it? Then the question turns int…

irrelevant points are "moot", not "mute"
Post reply on HN