For folks here, my RDS instances in us-east-1f are doing okay (knock on wood!) Not sure which AZ is suffering most. My client's Heroku instances are online, thankfully. Can anyone here speak to their experience with the Ohio region? I'm considering leaning on that more and more.
AWS EC2/RDS Outage in us-east-1
121–130 of 152 posts
Re: AWS EC2/RDS Outage in us-east-1
#122Earlier quoted context omitted.
In a world where you have SLAs with your customers, in which you commit to something better?
Damn, these ships must really be run tightly. In every company I have worked for, the amount of outages caused by bugs and other post deployment issues was already above that number.
Re: AWS EC2/RDS Outage in us-east-1
#123Earlier quoted context omitted.
For us this is exactly the correct approach. We could have spent millions of dollars and thousands of man hours hardening things to be resilient to single region outages. But for what? We aren't GE or Google. If our conference line goes down for 2-3 hours per year because we don't have apocalypse-proof infrastructure, literally nothing bad happens to our business. In this exact outage we are discussing, all of my cow…
2-3 hours per year is a lot of downtime. Most competent bare metal providers see maybe one major outage of less than an hour every 3-5 years. Nothing other than a facility wide power outage, if the load somehow gets dropped because the generators don't start right away as they should, or a misbehaving (only partially failing) core network infrastructure device should result in major outages when all the proper redund…
Re: AWS EC2/RDS Outage in us-east-1
#124Earlier quoted context omitted.
2-3 hours per year is a lot of downtime. Most competent bare metal providers see maybe one major outage of less than an hour every 3-5 years. Nothing other than a facility wide power outage, if the load somehow gets dropped because the generators don't start right away as they should, or a misbehaving (only partially failing) core network infrastructure device should result in major outages when all the proper redund…
I don't want to advertise your particular company, but if we are talking about numbers, how does your bare metal offer compares to a Amazon ec2 offer for example? And how would a customer that need to scale their load do it?
The CPU model is non-standard, but at 10 cores and 2.90GHz is effectively a slightly higher clocked version of the E5 2660v3 (10 cores, 2.60GHz). The first google result for a 2660v3 dedicated server with an order page that allows to adjust the options (13 usable IP's, 64GB RAM, minimal 120GB SSD storage) comes out to $275.
And this is based on whole box to whole box comparison. The cost of individual instances at AWS equivalent to one of those boxes can be much higher depending on the type and size.
Re: AWS EC2/RDS Outage in us-east-1
#125Earlier quoted context omitted.
Did you ever faced a situation where you or your clients needed more compute power and a cloud scalability would've been more convenient/cheaper?
No, a datacenter holds a seriously large amount of compute. Just a single rack is 38U usable in most cases, depending on density of compute and power availability you can get a good 2,000 CPU cores and a few dozen TiB of DDR4 from a single rack (with something like a DELL MX7000 chassis). And it's incredibly rare you'd be limited to a single rack of course. Cloud has many tangible benefits, but "amount of compute ava…
Re: AWS EC2/RDS Outage in us-east-1
#126For folks here, my RDS instances in us-east-1f are doing okay (knock on wood!) Not sure which AZ is suffering most. My client's Heroku instances are online, thankfully. Can anyone here speak to their experience with the Ohio region? I'm considering leaning on that more and more.
Your us-east-1f is not the same one as on other accounts. The letter is randomly assigned to the AZ to spread load.
Re: AWS EC2/RDS Outage in us-east-1
#127Earlier quoted context omitted.
2-3 hours per year is a lot of downtime. Most competent bare metal providers see maybe one major outage of less than an hour every 3-5 years. Nothing other than a facility wide power outage, if the load somehow gets dropped because the generators don't start right away as they should, or a misbehaving (only partially failing) core network infrastructure device should result in major outages when all the proper redund…
Is 3 hours per year really that bad? It’s 99.96% (three 9’s) uptime which I’d think is fine for most small to medium businesses.
Re: AWS EC2/RDS Outage in us-east-1
#128The Spinnaker project is looking more appealing with every outage. Outage detected in X provider in Y region? Deploy infrastructure to Z provider in Y region.
Who cares about stateless, that's a solved problem.
Re: AWS EC2/RDS Outage in us-east-1
#129Earlier quoted context omitted.
This should always be a business calculation but I do think you should note that there’s at least an order of magnitude difficulty increase between multiple AZs and regions, especially if you’re using services like RDS where it’s designed in, so I’d consider that a solid bridge step. I’m trying to put some numbers into that, I’ve been running a relatively well trafficked website in multiple AZs since 2011. We had ~20…
Good point, the multi-AZ RDS feature is a nice way to get most of the resilience upsides without any additional app complexity. You do double your database cost, but that might be worth it.
Re: AWS EC2/RDS Outage in us-east-1
#130Earlier quoted context omitted.
No, a datacenter holds a seriously large amount of compute. Just a single rack is 38U usable in most cases, depending on density of compute and power availability you can get a good 2,000 CPU cores and a few dozen TiB of DDR4 from a single rack (with something like a DELL MX7000 chassis). And it's incredibly rare you'd be limited to a single rack of course. Cloud has many tangible benefits, but "amount of compute ava…
Cloud can often be cheaper than on prem or at a colo if you are both willing to be “cloud native” and change your processes and you have people who actually know what they are doing and not a bunch of “lift and shifters” who are old school netops guide, got one AWS certification and now only know how to click around in the UI and duplicate an on prem infrastructure.