Live data from Hacker News

Securing a Postgres Database

goteleport.com

101–104 of 104 posts

Re: Securing a Postgres Database

#101

Possibly stupid questions. (1) When is operating your own PostgreSQL instance desirable? (2) Isn’t this equivalent to running an RDS instance in its own VPC with public access turned off and only allowing comms with app VPC?

Running your own Postgres is surprisingly easy, even with backups/monitoring/replication. Postgres replication has a reputation for being difficult to set up, but that's not what I've found. I think the reputation largely reflects how it was 15-20 years ago, not today. It's not completely turn-key I suppose, but there are many resources to learn.

Speaking of monitoring: there are far fewer options if you're on RDS. Personally my favorite is munin, where you can instantly see all kinds of stats & history at multiple levels of abstraction. There are many excellent Postgres plugins to report on transactions, locking, etc., and it's easy to write your own.

On RDS you can't install custom extensions. They have a whitelist of the most commonly-used ones, but if you find a different one (or build your own), you're out of luck. This really hampers you if you want to get the most from your database. I will say though, building custom extensions can also block you from using CI/CD solutions (or at least make them harder to set up), since they may have similar restrictions. Writing a custom extension is sort of a last restort, but it can be a huge boost for certain problems.

RDS also doesn't grant you direct access to the WAL. That means you can't use WAL-E/WAL-G (a really nice incremental backup solution) or many other helpful tools. You can't do replication except via AWS's own black-box features. (This can be especially annoying when you do upgrades.) It also matters because RDS only gives you 30 days of backups. Tons of businesses need more than that, and it's hard to achieve without using pg_dump. But for large databases, pg_dump can take hours and impact performance of other connections.

On RDS you also have to deal with EBS expense and performance limitations. PIOPS are very expensive. Running on local disks is a lot faster. On plain EC2 you can do more to work around all that. Ephemeral storage gives you real disks, but they might not be big enough (and they don't scale independently of the instance size). A better approach is RAIDing over gp2 volumes. I've heard they did this at Reddit and Citus. I've set it up before and it has worked great. You might be able to find some details in my comment history. Go back a few years. . . . Of course if running in your own datacenter is an option, that's even simpler (in some dimensions anyway). To me the sweet spot is renting dedicated machines, e.g. from Wholesale Internet.

RDS is sooo easy though. I have to admit it's hard not to recommend it for early-stage ventures. The limitations probably won't bite you until you're far along.

Re: Securing a Postgres Database

#102
post #88
post #78

Earlier quoted context omitted.

You can do the slow pbkdf2 hash on your webserver and use the result for your stored procedure to check the password. If the password column is only visible to the stored procedure you have used the best features of both systems.

No you can’t. To do the hash you need the per user salt before you start hashing. Which would require reading from the DB.

There is no reason you can‘t have a deterministic function which generates a hash for the user.

A simple sha256(lower(email)) is equally secure as a complete random salt, the only requirement on a salt is to be unique.

Re: Securing a Postgres Database

#103
post #99

Earlier quoted context omitted.

Usually application uses connection pool to connect with database server. I'm not sure if that pattern could be used, when different requests need different database roles.

It's doable. You have a web role in DB that's used for public facing part of a website and then admin role that's used for backoffice part of the website. You can then separate these backends to different machines or at least to a differnet UNIX users, so that when someone breaks through via public facing backend, he doesn't immediately get access to everything, but still has to work for it a bit. It's not as simple,…

Not to mention that with the microservices fad you could have different server processes with different credentials in their connection pools for different operations.
Post reply on HN