"Modern" architectures can get quite complex and fast at scale & in complex cases. This is merely a simple introduction to simple components of modern cloud architecture.
Modern cloud architecture on AWS: server fleets and databases
11–20 of 54 posts
Re: Modern cloud architecture on AWS: server fleets and databases
#12At what scale would you want to use RDS rather than using an EC2 instance with Postgres installed? Assuming that the operator has the skills to manage Postgres. It's not like RDS does something complex like Geodistribution, right? Also what is the scaling like? Is it automatic? How quickly can you handle more connections? Because my understanding was that it was slow. I did have a play with their RDS Postgres nonths…
I'd ask the opposite question - at what scale would you want to have your own custom setup rather than RDS? Managing your own database infrastructure for workloads other than "a few queries a second" is hard work with a lot of pitfalls, and you better be at a size that there's some benefit (high levels of customization, use case specific tuning, economies of scale, etc). As a person who does exactly this for a living, I'd rather shell out for RDS or a similar offering than my own setup most of the time. Especially at first, before you discover what exactly you /don't/ like about it or what you'd want different.
Re: Modern cloud architecture on AWS: server fleets and databases
#13I keep reading all these horror stories about Aurora (especially PostgreSQL). Is there anyone out there with an alternative story?
Using Aurora MySQL for over a year now in prod, purrs like a kittycat. Just don't use the AWS Database Migration Service if you can help it, that thing has a couple of badly documented pitfalls. (Fe. tables can't have ENUM fields)
AWS Database Migration Service had a shockingly large gap between how it's marketed and how well it actually performed.
It had so many gotchas and broken features. I'd be amazed if anyone got it really working on large applications without weeks of time invested.
It would be so cool though if it worked seamlessly. It addresses some of the hardest tasks in DB management, namely zero downtime server migration.
Re: Modern cloud architecture on AWS: server fleets and databases
#14"Modern" architectures can get quite complex and fast at scale & in complex cases. This is merely a simple introduction to simple components of modern cloud architecture.
I do agree with you, and I was lured into reading it because of that. However, this seems like a nice introduction for beginners. Maybe it should be tagged as such.
Re: Modern cloud architecture on AWS: server fleets and databases
#15At what scale would you want to use RDS rather than using an EC2 instance with Postgres installed? Assuming that the operator has the skills to manage Postgres. It's not like RDS does something complex like Geodistribution, right? Also what is the scaling like? Is it automatic? How quickly can you handle more connections? Because my understanding was that it was slow. I did have a play with their RDS Postgres nonths…
I'd ask the opposite question - at what scale would you want to have your own custom setup rather than RDS? Managing your own database infrastructure for workloads other than "a few queries a second" is hard work with a lot of pitfalls, and you better be at a size that there's some benefit (high levels of customization, use case specific tuning, economies of scale, etc). As a person who does exactly this for a living…
Is it hard work though? In a couple hours you should be able to setup automatic backups and practice going through the recover process a couple times. That's all there is for most small-business setups, but if you are daring you can now do whatever you want with the config file, install extensions, setup basic system monitoring (CPU/Ram usage, disk usage, etc.). GCP/Digital Ocean let you look at node resource usage automatically, and since Postgres is probably the only process it means you don't even need to set that up!
Re: Modern cloud architecture on AWS: server fleets and databases
#16At what scale would you want to use RDS rather than using an EC2 instance with Postgres installed? Assuming that the operator has the skills to manage Postgres. It's not like RDS does something complex like Geodistribution, right? Also what is the scaling like? Is it automatic? How quickly can you handle more connections? Because my understanding was that it was slow. I did have a play with their RDS Postgres nonths…
I'd ask the opposite question - at what scale would you want to have your own custom setup rather than RDS? Managing your own database infrastructure for workloads other than "a few queries a second" is hard work with a lot of pitfalls, and you better be at a size that there's some benefit (high levels of customization, use case specific tuning, economies of scale, etc). As a person who does exactly this for a living…
I used both too and I think it really depends if you want to pay for it. For small to medium independent projects, I think a EC2 instance that sometimes cuts past that free tier usage is fine, otherwise RDS can be overkill and can seriously eat into costs. I would say a beginner doing independent projects should strongly consider EC2 instances instead of RDS.
Re: Modern cloud architecture on AWS: server fleets and databases
#17At what scale would you want to use RDS rather than using an EC2 instance with Postgres installed? Assuming that the operator has the skills to manage Postgres. It's not like RDS does something complex like Geodistribution, right? Also what is the scaling like? Is it automatic? How quickly can you handle more connections? Because my understanding was that it was slow. I did have a play with their RDS Postgres nonths…
RDS (postgres, mysql, maria) is basically just a managed ec2 instance. The instance cost is about ~2x the comparable price for an ec2 instance, which seems reasonable to me. Storage costs are pretty comparable to EBS. You can do push-button upgrades to increase the capacity, but it's slow. The main benefits I think you get over self-managed is automated / on-demand backups, pretty seamless software upgrades/patches, you can quickly spin-up a duplicate instance from a snapshot for testing/distributing workloads/etc, and replication works pretty seamlessly.
Provisioned IOPS is one area that can get expensive very quick, but people often don't realize that you get 3 PIOPS included with every 1GB of allocated storage, so you really don't need to pay for provisioned IO if you have a decent amount of storage.
If you want auto-scaling you need to look at Aurora or Redshift, which are quite different and significantly more expensive. I've not used those.
Re: Modern cloud architecture on AWS: server fleets and databases
#18I keep reading all these horror stories about Aurora (especially PostgreSQL). Is there anyone out there with an alternative story?
Used it a year at my last job and now almost a year at my current job. Never had an issue. It just runs no matter how much I throw at it. Only have had to change the instance sizes to deal with data ingest.
Re: Modern cloud architecture on AWS: server fleets and databases
#19At what scale would you want to use RDS rather than using an EC2 instance with Postgres installed? Assuming that the operator has the skills to manage Postgres. It's not like RDS does something complex like Geodistribution, right? Also what is the scaling like? Is it automatic? How quickly can you handle more connections? Because my understanding was that it was slow. I did have a play with their RDS Postgres nonths…
For Cronitor, we did it when we needed replication.
Re: Modern cloud architecture on AWS: server fleets and databases
#20As someone who has spent a fair amount of time working with AWS. I appreciate how approachable this tutorial is, as the official docs are usually way more arcane.