Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we built yet another Postgres connection pooler

pgdog.dev

Post reply on HN
Print view View original

61–70 of 74 posts

Re: Why we built yet another Postgres connection pooler

#61

Post by alecco »

slopinthebag wrote: post #51
alecco wrote: post #49

Earlier quoted context omitted.

It is quite telling that most cloud companies stay away from AGPL. Their business model is antisocial.

You mean the licence is anti-business and pro-social.

AGPL only closes the SaaS loophole and forces them to publish their changes to the code. It's not anti-business at all.

You could argue the BSL is anti-business. But IMHO even that one is only a reaction to abuse. Database companies investing millions in research, development, and maintenance for some trillion dollar corporation to take it and make billions off it without giving back a single penny.

Re: Why we built yet another Postgres connection pooler

#63

Post by kardianos »

What posgresql needs is a new wire format. (pgwire4)

I've been slopping together a POC to probe the edges of what can be done as just an extension. So far I have a framed protocol with inline cancellation, named parameters, out-of-query text language selector, ad-hoc pg/PLSQL execution with cache (no need for prepare), multiple result sets, streaming large results, and more flexible bulk upload.

In other words, with this extension you can query:

``` select * from T1; select * from T2; ```

And return them both in PG/PLSQL or straight SQL.

The existing pgwire3 protocol is one of the worst things to work with in postgresql.

Re: Why we built yet another Postgres connection pooler

#64

Post by apt-get »

rubenvanwyk wrote: post #44

What really interests me most is the sharding and the possibility of using this for multitenancy - is the hooks / plugin architecture sufficient so you can run a small sidecar to add shards or tenants to the TOML file dynamically? Would be a game changer.

We found it pretty easy to build a little k8s controller for our own purposes to do this -- see https://news.ycombinator.com/item?id=48478994 . You probably don't need to implement this as a plugin or hook, pgdog supports dynamic reload of its configuration without dropping existing connections.

Although I'm the type to shy away from adding extra layers in my architectures when I can help it, pgdog has been an absolute breeze to use :)

Re: Why we built yet another Postgres connection pooler

#65

Post by ransom1538 »

Can someone explain it to me like I am 5. Why did postgres win vs mysql? I don't know many companies at scale that use postgres. Slack, youtube, etc all use a mysql based sharding system https://vitess.io/. I thought the war was lost for postgres, but it seems to keep going. License issues? (Disclosure, I manage 'a few' mysql vms).

Re: Why we built yet another Postgres connection pooler

#66

Post by re-thc »

ransom1538 wrote: post #65

Can someone explain it to me like I am 5. Why did postgres win vs mysql? I don't know many companies at scale that use postgres. Slack, youtube, etc all use a mysql based sharding system https://vitess.io/ . I thought the war was lost for postgres, but it seems to keep going. License issues? (Disclosure, I manage 'a few' mysql vms).

> Why did postgres win vs mysql?

They did? By social media? According to a lot of social media devs Java is also “dead”.

Having said that the issue is MySql / Mariadb is moving more and more behind commercial products e.g. Galera and Heatwave. Postgres continues to be the open community effort.

But hence the divide you see. Large companies. Real traffic use pragmatic solutions to make money. The tutorial developers and hype does whatever.

Re: Why we built yet another Postgres connection pooler

#67

Post by anticorporate »

ransom1538 wrote: post #65

Can someone explain it to me like I am 5. Why did postgres win vs mysql? I don't know many companies at scale that use postgres. Slack, youtube, etc all use a mysql based sharding system https://vitess.io/ . I thought the war was lost for postgres, but it seems to keep going. License issues? (Disclosure, I manage 'a few' mysql vms).

I don't understand the idea of "winning" in this context.

If your situation doesn't require specialized features of a particular database, then it doesn't really matter. Just pick one. There's too much premature optimization in the world.

If your situation does require something that one database or another excels at, then be grateful that there isn't really such thing as a winner and you can pick the one that works best for your context.

Re: Why we built yet another Postgres connection pooler

#68

Post by khurs »

kardianos wrote: post #63

What posgresql needs is a new wire format. (pgwire4) I've been slopping together a POC to probe the edges of what can be done as just an extension. So far I have a framed protocol with inline cancellation, named parameters, out-of-query text language selector, ad-hoc pg/PLSQL execution with cache (no need for prepare), multiple result sets, streaming large results, and more flexible bulk upload. In other words, with…

Have you raised it main developers and gained feedback?

Re: Why we built yet another Postgres connection pooler

#69

Post by Maledictus »

ransom1538 wrote: post #65

Can someone explain it to me like I am 5. Why did postgres win vs mysql? I don't know many companies at scale that use postgres. Slack, youtube, etc all use a mysql based sharding system https://vitess.io/ . I thought the war was lost for postgres, but it seems to keep going. License issues? (Disclosure, I manage 'a few' mysql vms).

MySQL is owned by the lawn mover* and barely kept alive.

* https://www.youtube.com/watch?v=-zRN7XLCRhc

Re: Why we built yet another Postgres connection pooler

#70

Post by Maledictus »

levkk wrote: post #18
mmakeev wrote: post #2

we moved our django app behind pgbouncer transaction pooling a few days ago and the surprise wasn't SET so much as queryset.iterator(). it relies on server side cursors, which don't survive being pooled, so we had to disable it everywhere and let it fall back to client side. also had to move statement_timeout out of the app's connection options into the pooler's own connect query, since libpq startup params just get…

Handling cursors is tough - they are very much session-level objects, so even if we, say, pinned your client while it uses that cursor, which would work, that would decrease the performance of connection pooling overall. So, what's better, breaking your app initially so you know to remove that feature, or letting it work silently while the connection pool isn't 100% in transaction mode? Tough call.

Which one did you pick?
Post reply on HN