Curious why this isn't a priority for the Postgres team. Not my area of expertise, and even I know about pgbouncer. Now another tool, so it seems like there's demand for it.
Odyssey: Scalable PostgreSQL Connection Pooler
11–20 of 54 posts
Re: Odyssey: Scalable PostgreSQL Connection Pooler
#12Are there benchmarks comparing this to Hikari? What is the benefit to using this over Hikari?
Hikari is a Java ODBC connection pooling library, meant to be used in your apps. Most other languages/frameworks also have client drivers that implement connection pooling today, but that only affects that specific program.
Odyssey / PGBouncer / PGpool are all external programs that implement connection pooling so multiple programs and servers can pool connections to a single database.
Re: Odyssey: Scalable PostgreSQL Connection Pooler
#13Can it do load balancing and failover?
No fail-over supported so far.
Re: Odyssey: Scalable PostgreSQL Connection Pooler
#14Curious why this isn't a priority for the Postgres team. Not my area of expertise, and even I know about pgbouncer. Now another tool, so it seems like there's demand for it.
Mostly because it's a lot of work due to Postgres process-per-connection architecture and since these projects already exist, so dev time is better spent elsewhere for now.
PG generally depends on 3rd-party tools to complete and polish the experience for those that need it.
Re: Odyssey: Scalable PostgreSQL Connection Pooler
#15Is this an alternative to PGBouncer, and if so, how do they compare?
This would be an alternative to pgbouncer, this github issue has a rundown of some of the differences - https://github.com/yandex/odyssey/issues/3
Re: Odyssey: Scalable PostgreSQL Connection Pooler
#16Curious why this isn't a priority for the Postgres team. Not my area of expertise, and even I know about pgbouncer. Now another tool, so it seems like there's demand for it.
There’s Postgres Pro Enterprise which is a commercial fork that has a connection poller feature.
Back in May it seemed to outperform Odyssey, according to a benchmark by one of the Postgres Pro folks: https://github.com/postgrespro/postgresql.builtin_pool/wiki/...
Re: Odyssey: Scalable PostgreSQL Connection Pooler
#17Looks cool. Maybe I'm missing something though, but it's not really clear how you'd set this up? Is there a quick start or some examples of configs for different situations?
I think it's a connection pooling proxy for postgres, and you talk to it using the regular postgres client protocol?
https://github.com/yandex/odyssey/blob/master/test/odyssey/t...
Re: Odyssey: Scalable PostgreSQL Connection Pooler
#18Curious why this isn't a priority for the Postgres team. Not my area of expertise, and even I know about pgbouncer. Now another tool, so it seems like there's demand for it.
The reason for it is because PostgreSQL wants to concentrate on correctness and do one thing and do it well. They will add a feature if there is a good reason for it and they find an elegant solution.
Having said that pooler is needed in all databases not just PostgreSQL. An evidence for it is just recently released MySQL proxy service in AWS. Here's[1] another database independent pool sizing guide with demo video of Oracle.
You are aware of pgbouncer mainly because of its feature which is transaction pooling, where each transaction is treated independently, it allows you to have thousands connections open, at the cost of not being allowed to use any session specific features of postgresql. This mode is really good for web applications and can work well even if your pool on application side is misconfigured.
[1] https://github.com/brettwooldridge/HikariCP/wiki/About-Pool-...
Re: Odyssey: Scalable PostgreSQL Connection Pooler
#19Curious why this isn't a priority for the Postgres team. Not my area of expertise, and even I know about pgbouncer. Now another tool, so it seems like there's demand for it.
There is definitely a demand for it, there are really three problems that exist with connections in Postgres.
1. The initial time to establish a connection is higher than would be ideal
2. Each connection, even an idle one, has an overhead of roughly 10 MB. For idle connections this is pretty much wasted space and most application frameworks grab a pool of connections accumulating quite a few idle connections.
3. The max number of connections you can push the system to is somewhere in hundreds to low thousand. Yes, it is possible to push PG further but it is not trivial.
But, the sad reality is it isn't a trivial quick fix. There are a lot of foundational issues with connection that have to be fixed. There are some folks and efforts in the community working towards this, and in my opinion it is the single biggest improvement that could happen in Postgres over the next 2-3 years, so hopefully it will improve directly in core over time.
Re: Odyssey: Scalable PostgreSQL Connection Pooler
#20Curious why this isn't a priority for the Postgres team. Not my area of expertise, and even I know about pgbouncer. Now another tool, so it seems like there's demand for it.
There’s Postgres Pro Enterprise which is a commercial fork that has a connection poller feature. Back in May it seemed to outperform Odyssey, according to a benchmark by one of the Postgres Pro folks: https://github.com/postgrespro/postgresql.builtin_pool/wiki/...
Aurora PostgreSQL also has a connection pooler, but it will still lose performance when you open too many connections. No matter what is used you still need to limit number of open connections to the database, so a pooler like pgbouncer in transaction mode which can drastically reduce them might still be needed.