Live data from Hacker News

Migrating to Postgres

engineering.usemotion.com

71–80 of 278 posts

Re: Migrating to Postgres

#71

Feels like postgres is always the answer. I mean like there's gotta be some edge case somewhere where postgres just can't begin to compete with other more specialized database but I'd think that going from postgres to something else is much easier than the other way around.

I hear MySQL can be better for some workloads?

Re: Migrating to Postgres

#72
post #55

> By Jan 2024, our largest table had roughly 100 million rows. I did a double take at this. At the onset of the article, the fact they're using a distributed database and the mention of a "mid 6 figure" DB bill made me assume they have some obscenely large database that's far beyond what a single node could do. They don't detail the Postgres setup that replaced it, so I assume it's a pretty standard single primary an…

You don't even need to be that "modern." Back in 2010 I was working on a MySQL 5.x system with about 300 million rows on a dual Xeon box with 16 gigs RAM and a few hundred gigs of RAID 10. This was before SSDs were common. The largest table was over 100 million rows. Some migrations were painful, however. At that time, some of them would lock the whole table and we'd need to run them overnight. Fortunately, this was…

The improvements to migrations have been the biggest boon for running even modestly-sized Postgres DBs. It wasn't that long ago that you couldn't add a column with a default value without rewriting the whole table, or adding NOT NULL without an exclusive lock while the whole table was scanned. That becomes unfeasible pretty quickly.

Re: Migrating to Postgres

#73
post #44

Earlier quoted context omitted.

Every ORM is bad. Especially the "any DB" ORMs. Because they trick you into thinking about your data patterns in terms of writing application code, instead of writing code for the database. And most of the time their features and APIs are abstracted in a way that basically means you can only use the least-common-denominator of all the database backends that they can support. I've sworn off ORMs entirely. My applicati…

I use PG with Entity Framework in .NET and at least 90% of my queries don't need any PG-specific features. When I need something PG specific I have options like writing raw SQL queries. Having most of my data layer in C# is fantastic for productivity and in most cases the performance compared to SQL is negligible.

Entity Framework really is such a time saver. The PG adapter makes it a breeze not just with common queries, but also more recent stuff, like working with embeddings for vector search.

Re: Migrating to Postgres

#75

Earlier quoted context omitted.

They've been saying that for 3 years. We actually had a discount for being an early adopter. But hey its obvious Ive never used it and only heard of it.

The JOIN mode has been in preview for over a year and is slated for GA release within a few months. Which has been on their roadmap. The removal of the rust service is available in preview for Postgres as of 6.7.[1] Rewriting significant parts of a complex codebase used by millions is hard, and pushing it to defaults requires prolonged testing periods when the worst case is "major data corruption". [1]: https://www.p…

It is hard.

Harder than just doing joins.

Re: Migrating to Postgres

#76

Earlier quoted context omitted.

They've been saying that for 3 years. We actually had a discount for being an early adopter. But hey its obvious Ive never used it and only heard of it.

The JOIN mode has been in preview for over a year and is slated for GA release within a few months. Which has been on their roadmap. The removal of the rust service is available in preview for Postgres as of 6.7.[1] Rewriting significant parts of a complex codebase used by millions is hard, and pushing it to defaults requires prolonged testing periods when the worst case is "major data corruption". [1]: https://www.p…

They've had flags and work arounds for ages. Not sure what point you are trying to make? But like you said I've never used it, only heard of it lol.

Re: Migrating to Postgres

#77
It's wild and hilarious, how often startups and companies go for distributed databases like CockroachDB/TiDB/Yugabyte before they actually need distribution, this trends sucks. 100 million rows is nothing that a well-tuned Postgres or MySQL instance (or even read-replicated setup) can't handle comfortably. Scale when you hit the wall.

Re: Migrating to Postgres

#78
post #31

Earlier quoted context omitted.

Prisma is so bad... can you believe it's by far the most downloaded ORM in NPM?

Every ORM is bad. Especially the "any DB" ORMs. Because they trick you into thinking about your data patterns in terms of writing application code, instead of writing code for the database. And most of the time their features and APIs are abstracted in a way that basically means you can only use the least-common-denominator of all the database backends that they can support. I've sworn off ORMs entirely. My applicati…

Every ORM except Active Record is awful. Active Record is amazing.

Re: Migrating to Postgres

#79
post #55

> By Jan 2024, our largest table had roughly 100 million rows. I did a double take at this. At the onset of the article, the fact they're using a distributed database and the mention of a "mid 6 figure" DB bill made me assume they have some obscenely large database that's far beyond what a single node could do. They don't detail the Postgres setup that replaced it, so I assume it's a pretty standard single primary an…

Does mid six figure mean ~$500k? That sounds insane for a crud app with one million users. What am I missing?

I’ve seen startups with a thousand active users paying $50k/month (though that’s overall costs, not just db). It’s really easy to waste a lot of money doing nothing.

Re: Migrating to Postgres

#80
post #40

I've lost count of how many "Migrating from X to Postgres" articles I've seen. I don't think I've once seen a migrating away from Postgres article.

I have participated in a Postgres -> Clickhouse migration, but I haven't bothered writing an article about it.
Post reply on HN