Architectures for Mitigating AWS Outages
forelse.io
Architectures for Mitigating AWS Outages
1–10 of 17 posts
Re: Architectures for Mitigating AWS Outages
#2This is kind of... fluffy for an HN post. It makes no mention of distributing your application between availability zones as a first step, swats away at the issues involved in any of the suggestions, then ends with “just configure what you need to configure for RDS and S3 replication and restore the rest by hand from backup.”
Re: Architectures for Mitigating AWS Outages
#3This is kind of... fluffy for an HN post. It makes no mention of distributing your application between availability zones as a first step, swats away at the issues involved in any of the suggestions, then ends with “just configure what you need to configure for RDS and S3 replication and restore the rest by hand from backup.”
It’s especially unnecessary when AWS has considerably more information available with the right questions to ask:
Re: Architectures for Mitigating AWS Outages
#4This is kind of... fluffy for an HN post. It makes no mention of distributing your application between availability zones as a first step, swats away at the issues involved in any of the suggestions, then ends with “just configure what you need to configure for RDS and S3 replication and restore the rest by hand from backup.”
Author here - totally fair. It is pretty high level. Distributing across AZs is mentioned in the fact it only accounts for HA, not DR.
The intent of this post was mainly to counter the arguments I was seeing online that dismissed multi-region setups as a way to guard against AWS outages because "it's too hard." I merely wanted to point out there is a way to do it without it being super complicated, or super expensive. There seems to be a trend to build super fancy architectures and as a whole, many people have neglected the simple "good enough" options.
Re: Architectures for Mitigating AWS Outages
#5This architecture is flawed, your DNS does not seems very resiliant. Also good luck with DB replication across region and the nasty side effects you can get with out of sync data.
My advice is: have a well designed architecture in a single region with multiple AZ and you will cover most problems.
Re: Architectures for Mitigating AWS Outages
#6Poor man's architecture for mitigating outages: Avoid us-east-1 (N Virginia). It unequivocally is the fail-whale of AWS regions. Every service there seems to have abnormally high usage numbers and absurd amount of scale to deal with. As James Hamilton would like to remind us, at such scale, even rare events are frequent [0]. The curse of being the default region, I suppose?
us-east-2 (Ohio) and eu-west-1 (Dublin) are my go-to regions. Prices are the same and most new services (and new features) are almost always ready to go on launch days.
[0] https://perspectives.mvdirona.com/2017/04/at-scale-rare-even...
Re: Architectures for Mitigating AWS Outages
#7This architecture is flawed, your DNS does not seems very resiliant. Also good luck with DB replication across region and the nasty side effects you can get with out of sync data. My advice is: have a well designed architecture in a single region with multiple AZ and you will cover most problems.
Flawed how? Route53 claims 100% uptime. Though if goes down, options are limited anyway. Are you saying a read replica won't stay in sync? Lag time on replication tends to be low. Keeping databases in sync across regions is definitely difficult, so i recommended against that.
Being in multiple AZs alone seems to be insufficient more and more. So the point was that by at least pushing backups or replicas into another region, you have options to restore from them.
Re: Architectures for Mitigating AWS Outages
#8This architecture is flawed, your DNS does not seems very resiliant. Also good luck with DB replication across region and the nasty side effects you can get with out of sync data. My advice is: have a well designed architecture in a single region with multiple AZ and you will cover most problems.
Flawed how? Route53 claims 100% uptime. Though if goes down, options are limited anyway. Are you saying a read replica won't stay in sync? Lag time on replication tends to be low. Keeping databases in sync across regions is definitely difficult, so i recommended against that. Being in multiple AZs alone seems to be insufficient more and more. So the point was that by at least pushing backups or replicas into another…
You could host your DNS elsewhere. Cloudflare (though they recently had a serious outage) or any of the million DNS providers that have been around forever.
Re: Architectures for Mitigating AWS Outages
#9Poor man's architecture for mitigating outages: Avoid us-east-1 (N Virginia). It unequivocally is the fail-whale of AWS regions. Every service there seems to have abnormally high usage numbers and absurd amount of scale to deal with. As James Hamilton would like to remind us, at such scale, even rare events are frequent [0]. The curse of being the default region, I suppose? us-east-2 (Ohio) and eu-west-1 (Dublin) are…
It's my understanding that they roll out changes there first, historically. New features land there first. Etc. US-EAST-1 IS the canary.
Re: Architectures for Mitigating AWS Outages
#10Poor man's architecture for mitigating outages: Avoid us-east-1 (N Virginia). It unequivocally is the fail-whale of AWS regions. Every service there seems to have abnormally high usage numbers and absurd amount of scale to deal with. As James Hamilton would like to remind us, at such scale, even rare events are frequent [0]. The curse of being the default region, I suppose? us-east-2 (Ohio) and eu-west-1 (Dublin) are…
It's my understanding that they roll out changes there first, historically. New features land there first. Etc. US-EAST-1 IS the canary.
If it's the canary, you don't want to be there if you're single region. You probably do want to be there if you're multi-region, in case they roll out something that breaks you, but not everyone else.