Live data from Hacker News

The startup's Postgres survival guide

hatchet.run

251–255 of 255 posts

Re: The startup's Postgres survival guide

#251
post #6

On migrations, there's a .Net tool called Grate that I tend to use for schema migrations... I don't use all the features, but it works well... using a migration stack in a repository for deployments and a similar tool is IMO more reliable than magic comparison tools or hand migrations in practice. You should defensively write your migrations as much as possible so that re-runs are relatively safe, though the tool hel…

> Even with indexes, joins are costly, especially under load at scale with millions of simultaneous users. You can avoid a lot of this by simply having that sub-table information inside a JSON field with the row in question. They’re really not that bad. Even on large-ish tables (hundreds of millions of rows), the typical query time I see for a query with 1-2 inner joins is 1-2 msec. That can of course vary with resul…

yeah, 1-2 across an index is fine... but I've seen live websites with very normalized structures where it took no less than 30 joins to get a flat view of a single resource. I could do the same with about 4 secondary queries (each with several joins), but I was trying to map a mental model of where all the data lived as a single view, so I could extract flat records for a search database.

There was absolutely zero advantage in how the data was structured in that case (auto classifieds website).

That's not to say that database normalization and modest level of joins is always bad... it's just something that can go too far, and holds things back even more in certain environments. FWIW, I started bundling child data in XML before JSON was a thing in highly controlled environments where any schema change was very difficult and required approvals through several groups in a large banking org.

That JSON is fairly well supported today only makes this an easier and better option IMO. Again, sometimes, but enough times to make note of.

Re: The startup's Postgres survival guide

#252

Earlier quoted context omitted.

How do you handle schema changes after your project is in production? I mean, sure start with a unified schema file until you have a production release... deploy, populate with placeholder data, etc... but once released, having a file for each set of changes isn't a bad thing. Also, the management tools you can have single files for each view/sproc, etc... it's just schema migrations you need to take care of.

Make a PR that edits the .sql file, deploy to staging, deploy to prod. Git tracks changes to the file, and your CI should be aware of what commit it's on. (If you even have CI) This only works if you don't care about being able to auto roll back DB changes without making a new commit, cause Postgres doesn't have a declarative DDL.

I'm more fond of just number+label files and a simple log as deployed to prevent re-run scenarios... Grate/Roundhouse does that for me... I don't use all the features, just separate each sproc/view and the up directory of migrations. This is checked into source control and gets deployed with the services themselves.

Re: The startup's Postgres survival guide

#253
post #62
post #49

Earlier quoted context omitted.

I might get flak for saying this but if you aren't a postgres expert already: just use RDS or a similar cloud DB. The amount of money you're saving by hosting and managing your own postgres instance is absolute peanuts compared to having battle-tested infrastructure for HA, backup and restores, point-in-time recovery, read replicas, etc.

At $dayjob we have the same mentality and as a result have a load of managed read replicas that are never used for anything (not reporting, not read only queries, not backups because $cloud handles it) that cost every month. Plus managed database restricts what you can do with the database - sometimes in really annoying ways. So while I partly agree with you, a lot of companies don't really need HA, read replicas, or…

That's really helpful. Thanks!

Re: The startup's Postgres survival guide

#254

Earlier quoted context omitted.

Cheapest Amazon RDS Postgres is like $15/mo if that's cheap enough. If you're doing lots of projects and don't want to pay that for each one, you can CREATE DATABASE for each with separate ACLs. You mentioned not wanting to spend admin hours fine-tuning a self-hosted instance though. Are you really hitting the DB hard enough for that to matter, but SQLite works fine? Cause I haven't tuned local Postgres in years.

Fair pushback, I think a lot of it is just stigma from not working close to the db layer and relying on these abstractions I think most of the work I've done just fits in the other storage models and they scale really far really fast at $0 cost I haven't paid for then ever, similarly, I have a few super tiny apps on Supabase and Neon too that cost me $0 On another project that got traction though, Postgres became bet…

Well it might be relational vs non-relational. And I get that SQLite is relational, but you might not be using it in a relational way.

Re: The startup's Postgres survival guide

#255
post #219

Earlier quoted context omitted.

Leaning SQL is arguably less dev work over the long run than learning an ORM and then learning how it works so you can fix performance issues.

Do you have any meaningful experience as a software developer? That’s a silly take. I somehow learned over the long run: SQL different flavors, couple of ORMs, multiple programming languages and multiple frameworks. Not counting different troubleshooting different operating systems and different applications I had to work with.

We all learn whatever we need to learn to deal with the situation. I've had to learn more distinct ORMs than SQL flavors, even though you also need to know the SQL when you use an ORM. Doesn't mean I wanted to.

However, each time a teammate wanted to use an ORM, I didn't call it silly or question their credentials.

Post reply on HN