AWS EC2/RDS Outage in us-east-1

Post by illumin8 » Mon, Sep 02, 2019, 7:06 AM UTC

dijit wrote: post #137

scarface74 wrote: post #133

Earlier quoted context omitted.

If all you care about is running a bunch of VMs yes. But if you’re just running a bunch of VMs and using cloud hosting - you’re doing it wrong. - we don’t want to maintain our own build servers. We just use CodeBuild with one of the prebuilt Docker containers or create a custom one. When we push our code to Github, CodeBuild brings up the Docker container, runs the build and the unit tests and puts the artifacts on S…

>I just don’t enjoy the infrastructure side that much. That's fair, that's totally your right. However, you're talking about absolute cost and unfortunately you're examples weave through true and false quite frenetically. > - we don’t want to maintain our own build servers. We just use CodeBuild with one of the prebuilt Docker containers or create a custom one. When we push our code to Github, CodeBuild brings up the…

> depending on your size of course, it could easily be cheaper to have a dedicated "build engineer" maintaining a build farm. This is how the majority of people do it. (I work in the video games industry, it's _MUCH_ cheaper to do it this way for us)

Cheaper to have a dedicated build engineer than using Codebuild? I just looked at my bill for August, my startup has $50K/mo. AWS spend across 4 regions in US/EU/Asia. We use Codebuild to build and deploy all of our infra from GitHub, including a ton of EC2 for our dedicated apps.

Guess how much my bill for Codebuild was in August? 17 cents! $0.17 CodeBuild $0.06 Asia Pacific (Singapore) $0.07 Asia Pacific (Tokyo) $0.02 EU (Frankfurt) $0.00 EU (Ireland) $0.02 US West (Oregon)

I'd like to see you hire a build engineer for $0.17. AWS services are dirt cheap because they let you automate all of the stuff that would require dedicated engineers for, while you can focus on your business, or what differentiates you.

Post by slisznia » Mon, Sep 09, 2019, 3:18 AM UTC

aeyes wrote: post #128

xyst wrote: post #28

The Spinnaker project is looking more appealing with every outage. Outage detected in X provider in Y region? Deploy infrastructure to Z provider in Y region.

Deploying a database with 5 TB on disk and 50k data-modifying transactions per second to multiple providers isn't exactly trivial. Who cares about stateless, that's a solved problem.

The volume and storage numbers you quoted call for breaking up a monolithic system. We have a similar overall volume of transactions and storage needs to yours but we reside in all US regions, tens of AZs. If one AZ went down, a small fraction of customers would be affected. True, we have increased our exposure to an outage happening somewhere but we improved overall business continuity. Simply put, dealing with 5% of pissed customers is easier than 50% or all of them calling you at once.

Re: AWS EC2/RDS Outage in us-east-1

Re: AWS EC2/RDS Outage in us-east-1