Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Migrating to Postgres

engineering.usemotion.com

Post reply on HN
Print view View original

121–130 of 278 posts

Re: Migrating to Postgres

#121

Post by wredcoll »

hu3 wrote: post #106
psionides wrote: post #50

Earlier quoted context omitted.

Yeah so there's basically just that one ;)

https://engineeringblog.yelp.com/2024/10/migrating-from-post...

You're not wrong, but this is the tldr:

> The DB used is PostgreSQL which is not used anywhere else at Yelp, which meant that only a small rotation of long-tenured employees knew Postgres well enough to do outage response. This caused issues in maintenance, visibility, and outage response times. The teams working on the Restaurants products are not infra teams, and the Yelp-wide infra teams (understandably) focus on Yelp-standard infrastructure. As a result, when we did see issues with Postgres it was often a scramble to find people with relevant expertise.

> So, we switched out this DB in-place with a Yelp-standard MySQL DB.

Re: Migrating to Postgres

#122

Post by marcus_holmes »

evantbyrne wrote: post #92
VWWHFSfQ wrote: post #38

Earlier quoted context omitted.

Every ORM is bad. Especially the "any DB" ORMs. Because they trick you into thinking about your data patterns in terms of writing application code, instead of writing code for the database. And most of the time their features and APIs are abstracted in a way that basically means you can only use the least-common-denominator of all the database backends that they can support. I've sworn off ORMs entirely. My applicati…

Nah. The most prolific backend frameworks are all built on ORMs for good reason. The best ones can deserialize inputs, validate them, place those object directly into the db, retrieve them later as objects, and then serialize them again all from essentially just a schema definition. Just to name a few advantages. Teams that take velocity seriously should use ORMs. As with any library choice you need to carefully vet…

The "good reason" is that modern web devs do not consider SQL a core skill, and plain do not understand databases. To be a competing modern web framework you have to include an ORM so these people will consider you.

Trying to explain to a modern web dev that the optimum data storage structure is not the same as the optimum application layer data structure, so you can't just take one and map them across 1:1 to the other, is really painful.

Developing without an ORM is just as quick as developing with one (because the time you save on routine queries you will more than lose on the tricky edge cases that the ORM completely screws up on). But you need to know SQL and databases to do it.

Re: Migrating to Postgres

#123

Post by sitharus »

seer wrote: post #103
pier25 wrote: post #31

Earlier quoted context omitted.

Prisma is so bad... can you believe it's by far the most downloaded ORM in NPM?

I don’t understand the hate, the only truly limiting factor for Prisma right now is its poor support for polymorphism, apart from that it has quite good support for complicated index setups, and if you need anything more performant, just drop to typed raw sql queries, it also supports views (materialized or otherwise) out of the box. I recently wanted to check it out and wrote a small app that had good use of pgvecto…

I don't hate prisma - it's just a tool - but that's far from the only limiting factor.

I recently looked at migrating a legacy project with basic SQL query generation to a modern ORM. Prisma came up top of course so I tried it.

We use Postgres built-in range types. Prisma does not support these, there's no way to add the type to the ORM. You can add them using "Unsupported", but fields using that aren't available in queries using the ORM, so that's pretty useless.

It also requires a binary to run, which would require different builds for each architecture deployed to. Not a big thing but it was more annoying than just switching the ORM.

That coupled with their attitude to joins - which has truth to it, but it's also short-sighted - eliminated Prisma.

The final decision was to switch to Kysely to do the SQL building and provide type-safe results, which is working well.

Re: Migrating to Postgres

#124

Post by pier25 »

bob1029 wrote: post #41
pier25 wrote: post #30

Earlier quoted context omitted.

> I can't stand ORMs, I don't get why people use it, please just write the SQL. I used to agree until I started using a good ORM. Entity Framework on .NET is amazing.

> Entity Framework on .NET is amazing. I disagree. It is probably one of the less terrible ORMs, but it is far from amazing. The object-relational impedance mismatch will always dominate for anything that isn't trivial business. EF works great until you need different views of the model. It does support some kind of view mapping technique, but it's so much boilerplate I fail to see the point. Dapper + SqlConnection i…

> I feel a lot more comfortable running manual schema migrations

Me too. I use a DB-first approach. Then EF simply rebuilds the application models automatically with a single command.

Re: Migrating to Postgres

#125

Post by rrr_oh_man »

sroussey wrote: post #16
amazingamazing wrote: post #9

Earlier quoted context omitted.

I don't disagree with your point, but over normalization and joining everywhere also isn't necessarily the answer, even with an index. there's no easy answer to this, really depends on the performance characteristics the critical user journeys need. with a little pain, if I had to pick an extreme, I'd pick extreme normalization with materialized views that are queried (e.g. no joins), rather than joining all of the t…

I typically go for 3rd normal form, and selectively denoralize where it has true performance value.

Just saying this feels like it elevates you above 80% of people who work with databases.

Re: Migrating to Postgres

#126

Post by rrr_oh_man »

reissbaker wrote: post #19
sgarland wrote: post #7

It is forever enraging to me that ORMs turn SELECT * into each individual column, mostly because people then post the whole thing and it’s obnoxiously large. Similarly maddening, the appalling lack of normalization that is simply taken for granted. “It’s faster, bro.” No, no, it is not. Especially not at the hundreds of millions or billions of rows scale. If you store something low-cardinality like a status column, w…

Eh, I've run applications on RDBMSes with multi-billion-row tables, and I've never found normalization/denormalization to be particularly impactful on performance except for in a few rare cases. The biggest impact came from sensible indexing + query patterns. Normalization vs denormalization had a big impact on convenience, though (not always favoring one way or the other!). But I'm no fan of Prisma either. Drizzle h…

> I've never found normalization/denormalization to be particularly impactful on performance

Really?

Re: Migrating to Postgres

#127

Post by hodgesrm »

I_am_tiberius wrote: post #101
jacobsenscott wrote: post #100

Earlier quoted context omitted.

CH excels at extremely high volume writes. You probably can't throw enough data at it.

Sorry, meant writes in terms of update/delete.

There are many use cases that do not alter data. Observability is a canonical example.

Re: Migrating to Postgres

#128

Post by [deleted] »

gbear605 wrote: post #79
thomasfromcdnjs wrote: post #66

Earlier quoted context omitted.

Does mid six figure mean ~$500k? That sounds insane for a crud app with one million users. What am I missing?

I’ve seen startups with a thousand active users paying $50k/month (though that’s overall costs, not just db). It’s really easy to waste a lot of money doing nothing.

[deleted]

Re: Migrating to Postgres

#129

Post by notTooFarGone »

etler wrote: post #40

I've lost count of how many "Migrating from X to Postgres" articles I've seen. I don't think I've once seen a migrating away from Postgres article.

We migrated from postgres to ADX based on cost analysis done of the managed version on Azure.

Now we have lovely kql queries and pretty much start new with postgres again...

Re: Migrating to Postgres

#130

Post by rakejake »

etler wrote: post #40

I've lost count of how many "Migrating from X to Postgres" articles I've seen. I don't think I've once seen a migrating away from Postgres article.

Probably a corollary of the fact that most usecases can be served by an RDBMS running on a decently specced machine, or on different machines by sharding intelligently. The number of usecases for actual distributed DBs and transactions is probably not that high.
Post reply on HN