Live data from Hacker News

Are triggers really that slow in Postgres?

cybertec-postgresql.com

11–20 of 79 posts

Re: Are triggers really that slow in Postgres?

#11
post #3

The biggest concern that engineering teams seem have with stored procs is maintainability. The consistency guarantees are indeed very attractive, but perhaps not at the expense of keeping business logic in separate places, weak version control/deploy solutions, and a new programming language to the stack. If there were a compiler that could take business logic in the project's programming language and manage mappings…

> weak version control/deploy solutions, and a new programming language to the stack

Are people not using the same version control for stored procedures as their "normal" programs? I thought deployment was pretty much a solved issue with DBAs.

Re: Are triggers really that slow in Postgres?

#12
post #3

The biggest concern that engineering teams seem have with stored procs is maintainability. The consistency guarantees are indeed very attractive, but perhaps not at the expense of keeping business logic in separate places, weak version control/deploy solutions, and a new programming language to the stack. If there were a compiler that could take business logic in the project's programming language and manage mappings…

There’s nothing stopping you from putting a script to create all of your triggers under version control. With your table and index creation scripts...

Re: Are triggers really that slow in Postgres?

#13

short answer: no, not slow. longer answer: you can do some really cool things with triggers in postgres, my favorite is what I like to refer to as "writeable views" - https://legitimatesounding.com/blog/stupid_postgresql_tricks... (2010)

FWIW since 9.6 this behavior is automatic for views with simple column references against a single relation

Re: Are triggers really that slow in Postgres?

#14
post #3

The biggest concern that engineering teams seem have with stored procs is maintainability. The consistency guarantees are indeed very attractive, but perhaps not at the expense of keeping business logic in separate places, weak version control/deploy solutions, and a new programming language to the stack. If there were a compiler that could take business logic in the project's programming language and manage mappings…

Interesting, I think that a good CI/CD story is probably the biggest thing holding broader use of stored procedures back.

that's part of the reason I built equinox: https://github.com/jerrysievert/equinox

if you're moving logic into your database, it's very important to be able to test and understand your use cases.

Re: Are triggers really that slow in Postgres?

#15
post #3

The biggest concern that engineering teams seem have with stored procs is maintainability. The consistency guarantees are indeed very attractive, but perhaps not at the expense of keeping business logic in separate places, weak version control/deploy solutions, and a new programming language to the stack. If there were a compiler that could take business logic in the project's programming language and manage mappings…

Interesting, I think that a good CI/CD story is probably the biggest thing holding broader use of stored procedures back.

This sort of sentence is uttered by someone who is so far down a hole that all they can see is stars. It’s a big hole that a lot of people are in but it’s the consequence of a nasty trade off.

All of this is trivial if your team decides that a shared database for dev work is bad for repeatability and thus bad for scaling the team.

You should be able to spool up a local database with good sample data in it. To do that your schema, indexes and triggers would be under version control, and a data dump is somewhere people can get it.

Once you have this your CI system runs Postgres locally or in a container, runs the drop create scripts, and then runs your integration and end to end tests on the canned data.

In addition to getting a CI solution for next to free you get rid of the concurrent access Wild West and this particularly painful conversation:

Why did this break and why didn’t you notice it before you pushed? Oh I saw that problem the other day but I thought someone else was changing data (and not my code being broken).

If the data is on your machine and it gets broken, then it is only your machine that could have broken it. You can’t delude yourself into thinking it was someone else mucking around. The problem is either in your code or in your latest pull from master. You are responsible for determining the cause, not me, not the release manager, not QA. You.

Re: Are triggers really that slow in Postgres?

#16
post #3

The biggest concern that engineering teams seem have with stored procs is maintainability. The consistency guarantees are indeed very attractive, but perhaps not at the expense of keeping business logic in separate places, weak version control/deploy solutions, and a new programming language to the stack. If there were a compiler that could take business logic in the project's programming language and manage mappings…

For a large node app, we wrote most of the business logic in JS and ran with PLV8. All the procedures were called from a single PLV8 library in shared memory, and we created functions stubs so you could still call with straight SQL (mostly used for ETL scripts).

Re: Are triggers really that slow in Postgres?

#17

The best way to discuss any speed thing is with anecdotes, right? /sarcasm I had a system that required the parsing of large json chunks. The system pulled the json from an API, pushed the data into a json-type column, then sorted the data into normal form. I originally tried using straight Python to pull the data, but decided that I ought to keep the original data for record keeping, plus testing was a lot faster wi…

I have no experience in using the json column in postgres (well, sort of did, but it was just a json type, before json column was a real thing).

For me, I try to do as much on the database side like sorting, which does require a good schema and table designs. This is why folks often criticize MongoDB. One of the reasons was the convincence of “schemaless”.

When Mongo was first introduced, I think a lot of developers, including me, saw Mongo as an excuse to move away from relational databases. So we began dumping all kinds of shit. Doing fancy stuff on Mongo side is not possible without a good design either.

What people probably did was just pulling data from multiple collections, and do filtering and “joins” on the server (client) side. I would find myself writing a for loop over doing a bunch of stuff. Yikes.

Of course there are other criticisms against MongoDB, but ultimately developers like myself did not (and probably still) have any decent clues how to use databases well. Learnig to use databases right is something I really want to be good at.

Re: Are triggers really that slow in Postgres?

#18
post #3

The biggest concern that engineering teams seem have with stored procs is maintainability. The consistency guarantees are indeed very attractive, but perhaps not at the expense of keeping business logic in separate places, weak version control/deploy solutions, and a new programming language to the stack. If there were a compiler that could take business logic in the project's programming language and manage mappings…

> weak version control/deploy solutions, and a new programming language to the stack Are people not using the same version control for stored procedures as their "normal" programs? I thought deployment was pretty much a solved issue with DBAs.

Based on the experience of co-workers (and my own limited dabbling), you are right that version control is (and should be) used for stored procedures just as it is with application code. The problem is that there isn't a lot of tooling that isn't language- or platform-specific to help maintain these things in the long run. So DBAs tend to write and maintain their own collection of homegrown scripts for deploying changes to the DB (or worse, do it by hand).

I'm sure some companies have this figured out, though, so YMMV.

Re: Are triggers really that slow in Postgres?

#19

short answer: no, not slow. longer answer: you can do some really cool things with triggers in postgres, my favorite is what I like to refer to as "writeable views" - https://legitimatesounding.com/blog/stupid_postgresql_tricks... (2010)

Fantastic feature that makes it trivial to write several different types of backwards compatible DDL changes.

For instance, you want to rename a column? Add the new column, rename the existing table, add a view in its place with both the old column name and the new column name, with an INSTEADOF trigger to update the base table. Postgres lets you do this all in a transaction, so the entire operation is atomic.

Re: Are triggers really that slow in Postgres?

#20
post #3

The biggest concern that engineering teams seem have with stored procs is maintainability. The consistency guarantees are indeed very attractive, but perhaps not at the expense of keeping business logic in separate places, weak version control/deploy solutions, and a new programming language to the stack. If there were a compiler that could take business logic in the project's programming language and manage mappings…

> weak version control/deploy solutions, and a new programming language to the stack Are people not using the same version control for stored procedures as their "normal" programs? I thought deployment was pretty much a solved issue with DBAs.

I suspect it’s the shared infrastructure problem. People who are used to running a microcosm of their app locally don’t usually make statements like that in my experience.

I have a dev box with 16 GB of RAM that sits unused all day because I work with people who think this way. Such a waste of resources.

Post reply on HN