Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pgagroal: High-performance connection pool for PostgreSQL

github.com

Post reply on HN
Print view View original

21–30 of 39 posts

Re: Pgagroal: High-performance connection pool for PostgreSQL

#21

Post by whitepoplar »

api wrote: post #18
gmac wrote: post #14

Earlier quoted context omitted.

First: have you used Postgres, and at what scale? Because that hasn't been my experience of it. Second: do you know something that's comparable but better?

I should have been more detailed. Postgres works really well in single node configurations as long as you read up a bit on the weirdness around vacuuming. Usually you don't have to worry unless you are running write-heavy loads at large scale. HA Postgres is where things get frustrating and hairy. It's 2021. I should be able to set up a master-master multi-node database by running a second node and telling it where t…

Check out pg_auto_failover! It's the easiest HA option for Postgres atm.

Re: Pgagroal: High-performance connection pool for PostgreSQL

#22

Post by api »

qaq wrote: post #12
api wrote: post #11

Earlier quoted context omitted.

The popularity and mindshare of Postgres really puzzles me. It’s so arcane and full of weird failure modes and gotchas. It feels like something from a 1980s mainframe that would come with a bound “field manual.” We can have file systems like zfs that for all but maybe the largest and most exotic use cases are bulletproof and relatively easy to administrate. Why can’t we have SQL like this? I guess we do in the form o…

You do realize that prob 80% of Yugobyte code is PostgreSQL?

Yes. The other 20% implements Raft consensus.

Re: Pgagroal: High-performance connection pool for PostgreSQL

#23

Post by Scarbutt »

api wrote: post #11
skrause wrote: post #4

"PostgreSQL's model of one process per connection doesn't scale, so let's build a connection pooler that uses one process per connection." Certainly sounds like a strange concept. How well does it scale with thousands of connections?

The popularity and mindshare of Postgres really puzzles me. It’s so arcane and full of weird failure modes and gotchas. It feels like something from a 1980s mainframe that would come with a bound “field manual.” We can have file systems like zfs that for all but maybe the largest and most exotic use cases are bulletproof and relatively easy to administrate. Why can’t we have SQL like this? I guess we do in the form o…

This forum is heavily anti-other-dbs and pro-postgres so you won't get objective answers here. FWIW, at least three DBAs that know both well have told me that postgres is an operational nightmare compared to mysql.

Re: Pgagroal: High-performance connection pool for PostgreSQL

#24

Post by djbusby »

rkrzr wrote: post #16
api wrote: post #11

Earlier quoted context omitted.

The popularity and mindshare of Postgres really puzzles me. It’s so arcane and full of weird failure modes and gotchas. It feels like something from a 1980s mainframe that would come with a bound “field manual.” We can have file systems like zfs that for all but maybe the largest and most exotic use cases are bulletproof and relatively easy to administrate. Why can’t we have SQL like this? I guess we do in the form o…

As a long-time happy user of Postgres, I am happy to answer your questions: > The popularity and mindshare of Postgres really puzzles me. Why? Postgres really is "The World's Most Advanced Open Source Relational Database", just like it says on the tin. > It’s so arcane and full of weird failure modes and gotchas. Not really. It actually is the best-documented open source database that behaves just like they say in th…

I've been using PG for like 20 years. Never paid, except for odd donation here and there, last trigger by the add of jsonb type.

Re: Pgagroal: High-performance connection pool for PostgreSQL

#25

Post by endisneigh »

Scarbutt wrote: post #23
api wrote: post #11

Earlier quoted context omitted.

The popularity and mindshare of Postgres really puzzles me. It’s so arcane and full of weird failure modes and gotchas. It feels like something from a 1980s mainframe that would come with a bound “field manual.” We can have file systems like zfs that for all but maybe the largest and most exotic use cases are bulletproof and relatively easy to administrate. Why can’t we have SQL like this? I guess we do in the form o…

This forum is heavily anti-other-dbs and pro-postgres so you won't get objective answers here. FWIW, at least three DBAs that know both well have told me that postgres is an operational nightmare compared to mysql.

In which ways?

Re: Pgagroal: High-performance connection pool for PostgreSQL

#26

Post by Scarbutt »

endisneigh wrote: post #25
Scarbutt wrote: post #23

Earlier quoted context omitted.

This forum is heavily anti-other-dbs and pro-postgres so you won't get objective answers here. FWIW, at least three DBAs that know both well have told me that postgres is an operational nightmare compared to mysql.

In which ways?

Don't know the exact details but they mentioned is full of operational footguns, like using it slightly wrong can utterly tank your performance or availability. Many operations are unpredictable so is hard to know what to expect.

Another reason they mentioned is InnoDB, in particular that is the most battle-tested and well-engineered storage engine. With enormous amounts of engineering effort in hardening it for performance and for stability and operability at scale by companies like Google and Facebook.

They also like Vitess.

Re: Pgagroal: High-performance connection pool for PostgreSQL

#27

Post by oliwarner »

bigbillheck wrote: post #20
maxpert wrote: post #9

> This report describe pgagroal in relationship to 3 other PostgreSQL connection pool implementations, which we will call "a", "b" and "c". Why? Just why! Please put up names rather than saving faces for other poolers!

> Why? Just why! Please put up names rather than saving faces for other poolers! It's Larry Ellison's fault: https://en.wikipedia.org/wiki/David_DeWitt

This is an [interesting] joke, right?

Re: Pgagroal: High-performance connection pool for PostgreSQL

#28

Post by bigbillheck »

oliwarner wrote: post #27
bigbillheck wrote: post #20

Earlier quoted context omitted.

> Why? Just why! Please put up names rather than saving faces for other poolers! It's Larry Ellison's fault: https://en.wikipedia.org/wiki/David_DeWitt

This is an [interesting] joke, right?

No? Oracle started putting clauses in their sales contracts limiting publishing benchmarks because they performed poorly in a published report.

Re: Pgagroal: High-performance connection pool for PostgreSQL

#30

Post by anarazel »

jpgvm wrote: post #5
skrause wrote: post #4

"PostgreSQL's model of one process per connection doesn't scale, so let's build a connection pooler that uses one process per connection." Certainly sounds like a strange concept. How well does it scale with thousands of connections?

Not the author but PG connection scalability being blamed on per-process model is somewhat misunderstood. You can easily have many hundreds of thousands of processes, they are mostly equivalent to threads on Linux anyway (with some minor differences ofc). Instead the poor scalability stems from the amount of resources allocated to each connection. If postgres was to switch to threads without changing this architectur…

> You can easily have many hundreds of thousands of processes, they are mostly equivalent to threads on Linux anyway (with some minor differences ofc).

Not really. The page-table overhead of a 100k processes alone is going to a eat up a lot of memory - and you don't have that with threads. And with something like postgres the page tables aren't going to be tiny, even with explicit use of huge pages for applicable allocations. The decrease in TLB hit ratio also is very painful. In a lot of workloads postgres spends way too much time in TLB faults, and that's to a good degree caused by the process model (also caused by linux making it way too hard to map executables with huge pages, itlb faults are a significant issue).

That's not to say that a 1 connection: 1 thread approach is the right thing for a pooler - threads area also don't scale great. I think there are good simplicity arguments to be made for a 1 connection: 1 thread model for a database server itself, but for poolers the benefits seem much smaller.

Post reply on HN