Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern cloud architecture on AWS: server fleets and databases

start.jcolemorrison.com

Post reply on HN
Print view View original

21–30 of 54 posts

Re: Modern cloud architecture on AWS: server fleets and databases

#21

Post by mdeeks »

Jonnax wrote: post #4

At what scale would you want to use RDS rather than using an EC2 instance with Postgres installed? Assuming that the operator has the skills to manage Postgres. It's not like RDS does something complex like Geodistribution, right? Also what is the scaling like? Is it automatic? How quickly can you handle more connections? Because my understanding was that it was slow. I did have a play with their RDS Postgres nonths…

I'll take a shot at this. There is always an asterisk under every one of these. Every company and situation is different.

If you're a tiny startup or hobby with literally no money, it might make sense for you to manage it yourself because you have no choice.

Once you have some money and a viable business, then your value is no longer your ability to spend your time running Postgres, ensuring backups and restores work, creating replicas, upgrading software, and setting up all of the monitoring tools. You provide MUCH more value spending your time and abilities building things that are core to the business that let you make money and grow.

No doubt you can do it all and save some cash. But you have to do it regularly if you want confidence that everything you have built still works. With RDS, you pay them some extra for a near guarantee that it will all just work 100% of the time.

Once you become a large company with tons of engineers and you start to bump into limits of RDS, then it might make sense to run it yourself again. It is a significant burden to do it correctly 100% of the time. Your entire business can fail if you don't do your job right.

Re: Modern cloud architecture on AWS: server fleets and databases

#22

Post by nunez »

Jonnax wrote: post #4

At what scale would you want to use RDS rather than using an EC2 instance with Postgres installed? Assuming that the operator has the skills to manage Postgres. It's not like RDS does something complex like Geodistribution, right? Also what is the scaling like? Is it automatic? How quickly can you handle more connections? Because my understanding was that it was slow. I did have a play with their RDS Postgres nonths…

You would move to RDS the minute you know you're committed to AWS and know that you don't want to worry about things like backups, upgrades or clustering. The disadvantage to doing so is that you lose a lot of administrative privileges to the database server itself and you don't get access to the filesystem or underlying OS. We had trouble migrating a sizable SQL Server installation onto RDS because it had a ton of triggers and stored procedures (SPs) that relied on files in the filesystem.

Re: Modern cloud architecture on AWS: server fleets and databases

#23

Post by reilly3000 »

Jonnax wrote: post #4

At what scale would you want to use RDS rather than using an EC2 instance with Postgres installed? Assuming that the operator has the skills to manage Postgres. It's not like RDS does something complex like Geodistribution, right? Also what is the scaling like? Is it automatic? How quickly can you handle more connections? Because my understanding was that it was slow. I did have a play with their RDS Postgres nonths…

RDS can scale read replicas and fail over to master, but aurora and it’s serverless option is much better for auto-scaling. Behind the scenes storage is decoupled from compute with makes scaling fundamentally easier.

EC2 is your only choice if you want a database that AWS doesn’t support, such as Rethink or Cassandra (they just recently launched a managed Cassandra service though). EC2 is also your only choice if you need full control of the DB, such as using many Postgres extensions and foreign data wrappers. Even some triggers and UDFs are limited.

A self-managed, auto-scaling, cross-AZ replicated DB setup is no small matter with EC2. Not to mention logging, metrics, patching of the DB and underlying OS. It’s 100% doable, but one should only proceed with that course with understanding of the human costs.

Personally, I’ve been choosing FaunaDB these days when possible. It’s a no-ops managed service and has on-prem/VPC options. I just write graphQL clients and move on with my life, the rest just works.

Re: Modern cloud architecture on AWS: server fleets and databases

#24

Post by nunez »

mdeeks wrote: post #21
Jonnax wrote: post #4

At what scale would you want to use RDS rather than using an EC2 instance with Postgres installed? Assuming that the operator has the skills to manage Postgres. It's not like RDS does something complex like Geodistribution, right? Also what is the scaling like? Is it automatic? How quickly can you handle more connections? Because my understanding was that it was slow. I did have a play with their RDS Postgres nonths…

I'll take a shot at this. There is always an asterisk under every one of these. Every company and situation is different. If you're a tiny startup or hobby with literally no money, it might make sense for you to manage it yourself because you have no choice. Once you have some money and a viable business, then your value is no longer your ability to spend your time running Postgres, ensuring backups and restores work…

If I were a "tiny startup", I would absolutely leverage RDS in the first instance. Last thing I want to spend time doing is setting us Postgres and accumulating tech debt from a bad or suboptimal install. There are RDS instances that qualify for Free Tier as well if you're running _really_ lean.

In fact, the only time that I would consider migrating to a DB on EC2 is if the database isn't supported by RDS (DB2, for example, though I think there are solutions for this now), the licensing for RDS makes it cost-prohibitive or if the client needed to be able to really tweak the database engine or daemons.

Re: Modern cloud architecture on AWS: server fleets and databases

#25

Post by malisper »

Jonnax wrote: post #4

At what scale would you want to use RDS rather than using an EC2 instance with Postgres installed? Assuming that the operator has the skills to manage Postgres. It's not like RDS does something complex like Geodistribution, right? Also what is the scaling like? Is it automatic? How quickly can you handle more connections? Because my understanding was that it was slow. I did have a play with their RDS Postgres nonths…

I've previously administrated a cluster Postgres instances with a total of about 1PB of data. My recommendation is that you should use RDS unless you have a reason not to.

RDS takes care of tons of administrative tasks such as backups, replication, failover, and database upgrades. Yes you can setup backups yourself, but the on going maintenance is going to be a pain. You need to deal with what happens when a backup fails, have a playbook for restoring from backup, cleanup your old backups, etc. These are tasks that are extremely dangerous if you get something wrong and they are completely taking care of for you by RDS.

Re: Modern cloud architecture on AWS: server fleets and databases

#26

Post by malisper »

Although there are a ton of AWS servers, there's only a few core services that I recommend:

  EC2 - You need a server.
  RDS - You need a database.
  S3 - You need to store files.
  Lambda - You are building an API with short lived requests.
These services are all very high quality and are excellent at what they do. Once you get outside of these core services, the quality quickly drops. You're probably better off using the non-AWS versions of those services.

For a few quick examples, you should be using Datadog over CloudWatch, Snowflake over Redshift or Athena, and Terraform over CloudFormation.

Re: Modern cloud architecture on AWS: server fleets and databases

#27

Post by malisper »

tmpz22 wrote: post #15
makmanalp wrote: post #12

Earlier quoted context omitted.

I'd ask the opposite question - at what scale would you want to have your own custom setup rather than RDS? Managing your own database infrastructure for workloads other than "a few queries a second" is hard work with a lot of pitfalls, and you better be at a size that there's some benefit (high levels of customization, use case specific tuning, economies of scale, etc). As a person who does exactly this for a living…

Is it hard work though? In a couple hours you should be able to setup automatic backups and practice going through the recover process a couple times. That's all there is for most small-business setups, but if you are daring you can now do whatever you want with the config file, install extensions, setup basic system monitoring (CPU/Ram usage, disk usage, etc.). GCP/Digital Ocean let you look at node resource usage a…

> In a couple hours you should be able to setup automatic backups and practice going through the recover process a couple times.

Unfortunately there's a lot more too it than that. You need to handle when the backup job fails or dies, have a process for deleting old backups, etc. Not just that, but if you have multiple Postgres instances, you need to do this work for each machine. I've seen first hand this kind of stuff become a huge distraction. It's often worth it to pay AWS a bit more in exchange to not worry about this stuff.

Re: Modern cloud architecture on AWS: server fleets and databases

#28

Post by reese_john »

malisper wrote: post #26

Although there are a ton of AWS servers, there's only a few core services that I recommend: EC2 - You need a server. RDS - You need a database. S3 - You need to store files. Lambda - You are building an API with short lived requests. These services are all very high quality and are excellent at what they do. Once you get outside of these core services, the quality quickly drops. You're probably better off using the n…

Just curious, why would you not recommend SQS ?

Re: Modern cloud architecture on AWS: server fleets and databases

#29

Post by tmpz22 »

malisper wrote: post #27
tmpz22 wrote: post #15

Earlier quoted context omitted.

Is it hard work though? In a couple hours you should be able to setup automatic backups and practice going through the recover process a couple times. That's all there is for most small-business setups, but if you are daring you can now do whatever you want with the config file, install extensions, setup basic system monitoring (CPU/Ram usage, disk usage, etc.). GCP/Digital Ocean let you look at node resource usage a…

> In a couple hours you should be able to setup automatic backups and practice going through the recover process a couple times. Unfortunately there's a lot more too it than that. You need to handle when the backup job fails or dies, have a process for deleting old backups, etc. Not just that, but if you have multiple Postgres instances, you need to do this work for each machine. I've seen first hand this kind of stu…

> Unfortunately there's a lot more too it than that.

Is there though? Consider what I would argue to be the "average" case:

* Your database never exceeds > 40% resource usage

* You service fewer then 1m queries/day

* You never burst more then 1k queries/minute

* You have a script tied to a cronjob that backs up the database, with basic error handling that sends you a Slack DM if it fails

* You have a script tied to a cronjob which deletes old backups, with basic error handling that sends you a Slack DM if it fails

What percentage of companies need more then that?

Re: Modern cloud architecture on AWS: server fleets and databases

#30

Post by malisper »

reese_john wrote: post #28
malisper wrote: post #26

Although there are a ton of AWS servers, there's only a few core services that I recommend: EC2 - You need a server. RDS - You need a database. S3 - You need to store files. Lambda - You are building an API with short lived requests. These services are all very high quality and are excellent at what they do. Once you get outside of these core services, the quality quickly drops. You're probably better off using the n…

Just curious, why would you not recommend SQS ?

I've never used SQS but IMO it seems inferior to Kinesis or Kafka. The two big reasons are that you can't have multiple consumers read from a single queue and once data leaves the queue, it's gone forever. Both Kinesis and Kafka let you have multiple consumers and configure a retention period for your messages.
Post reply on HN