Live data from Hacker News

Does anyone run Postgres without PgBouncer?

brandur.org

71–80 of 121 posts

Re: Does anyone run Postgres without PgBouncer?

#71
Totally agree. Postgres core should include connection pooling by default, and PgBouncer is probably the natural path to get there.

With PgBouncer addressing one of its biggest historical pain points - prepared statement support in transaction mode, in our managed Postgres offering, we’re increasingly seeing customers use the PgBouncer connection string by default for mose use-cases without running into any hiccups. That wouldn’t necessarily have been the case a few years ago.

PgBouncer is also battle-tested, widely validated, and offers a (surprising) level of configurability. You could also run a peered setup and make it multi-threaded, which is something I didn’t expect when I first came to know about it. https://news.ycombinator.com/item?id=48872874

Re: Does anyone run Postgres without PgBouncer?

#72

Earlier quoted context omitted.

It's a little more nuanced than that. At my company we run Kubernetes workloads where we have 20-30 or sometimes more pods connecting to a single Postgres instance, each pod handling hundreds of concurrent requests. If each pod hangs on to pooled connections for more than a few seconds, then you end up with quite a few Postgres processes and a lot of memory usage and process churn; we mitigate that by having a relati…

SRE here, that's a ton of pods and sounds like it's language or architecture if you are serving up so few requests per pod. We have Kubernetes here too for Java monoliths and we have 8 Pods serving more than that. Since Java has connection pooling, the overhead has never been enough to justify overhead of Pgbouncer for this Java app.

Our workloads are served by Go processes, so quite lean. But the workloads are relatively CPU-bound and memory-hungry.

Re: Does anyone run Postgres without PgBouncer?

#74
post #40
post #35

Earlier quoted context omitted.

I feel like there are a lot of use cases where I’d opt for SQLite and a lot of use cases where I’d opt for Postgres + PgBouncer. I’m curious what kinds of features push towards using Postgres alone over SQLite.

All of my personal apps use Postgres because: - types are lovely. We love types. SQLite’s default of non-strict typing is, to me, bananas. - SELECT DISTINCT ON is my ride-or-die - most importantly, I’m very comfortable in Postgres and the setup cost is basically zero (like SQLite) because Claude does it.

The setup cost is basically zero anyway. Add apt repo, apt-get install the correct version. Easy to run different versions at the same time too. Upgrading is annoying.

Re: Does anyone run Postgres without PgBouncer?

#75
post #62

Earlier quoted context omitted.

It's an ironic claim because the Oracle cloud has great support for the Oracle Database, which doesn't require "bouncers" or equivalent. Its support for server-side connection pooling, client side load balancing (SCAN) and horizontal scaling means it offers exactly what the author wants - a single URL that just magically works and scales to any amount of work or connections cheaply. https://docs.oracle.com/en/databas…

Slightly off topic, but a friend of mine worked on Microsoft SQL Server a ~decade ago, and I recall all sorts of conversations on types of optimizations they were doing that sounded truly magical compared to other databases at the time. There was a meaningful frustration on the team that their features weren't getting as much hype as the databases du jour. I think it's very relevant to consider that "enterprise" adja…

Yeah, that's a common sentiment. I regularly see people describe features commercial RDBMS' have shipped for years as if they were unmapped frontiers in computer science. Awareness of their capabilities is very low; I was once there too and remember being taken by surprise when I realized how far ahead of open source they truly are.

IMO startups can get edge by exploiting this information asymmetry. Spending a bit more to solve all your DB problems and buy productivity is a no brainer as they have VC funding but not enough time. A single bad DB outage can be the difference between beating a competitor or losing to them. Ditto for slowly shipping a feature because your senior dev is trying to implement their own message queue engine or other random thing that comes out of the box in other RDBMS engines.

The costs depend what you compare it to. People tend to overestimate it. Cost multipler in Azure is very roughly about 4x, it seems (caveat: am not a cloud pricing expert, comparisons may vary wildly). That doesn't include the cost of bouncers and other hacks that increase the Postgres cost, so it's artificially generous to PG.

If you want better features on the Postgres side then you might look at AlloyDB in Google Cloud which is only 2x cheaper on compute but where storage is actually ~3x more expensive!

The extra money buys you a lot. Not only far more features but you can provision a smaller database because the Oracle DB burst scales in response to load. You are only charged for the extra you use so you can provision for normal load without padding extra for emergencies or peaks. It's a genuine cluster that scales up writes more or less indefinitely without sharding if you design your schema right, that's synchronous multi-write master scaling too so its simple for apps. You don't face OpenAI style problems where the single Postgres master reaches its limits and the whole thing breaks requiring app redesigns. And you aren't just paying for an idle replica: all the capacity you buy can be used for queries. It also uses more efficient algorithms e.g. better MVCC with no autovacuuming problems. And a gazillion other things.

So I think you can easily argue that value delivered is much greater than 2x-4x. The capability gap is much larger than 4x. Especially if you're the sort of startup where a bored dev might start citing Postgres' limitations to justify inventing their own DB infra, or where you hit its scaling limits and have to rearchitect - if that happens you'll never recover the cost difference, Oracle will always be cheaper.

This happens because clouds don't charge databases at licensing+labor value+margin, prices are set at what the market will bear.

Re: Does anyone run Postgres without PgBouncer?

#76
post #61

Earlier quoted context omitted.

It's an ironic claim because the Oracle cloud has great support for the Oracle Database, which doesn't require "bouncers" or equivalent. Its support for server-side connection pooling, client side load balancing (SCAN) and horizontal scaling means it offers exactly what the author wants - a single URL that just magically works and scales to any amount of work or connections cheaply. https://docs.oracle.com/en/databas…

I do not think it is a mystery why people favor Postgres over OracleDB, is it?

Depends what you mean by people. Outside of the startup space you'll find commercial RDBMS everywhere, especially OracleDB. Not many are running banks or hospitals on Postgres.

Re: Does anyone run Postgres without PgBouncer?

#77

(I work on the postgres proxy layer at Neon) PgBouncer is entirely optional and it's not always the right choice. If you have a classical app (non serverless) and you can maintain a connection pool from your app, then I recommend avoiding pgbouncer. The benefits of pgbouncer mostly come from irregular client connections (too many, too much churn). If you don't have that problem, go direct to postgres. I'm exploring r…

And the only comment in this whole thread, who argues for a similar architectural alternative...is the ONLY one in the whole thread down voted. HN continue to excel in technical chops... https://news.ycombinator.com/item?id=49319988 https://news.ycombinator.com/item?id=49320460

Unless I'm missing something, your linked comments seem to be responding only to the title of the article, without addressing any of the actual points being made in the body of the article?

Re: Does anyone run Postgres without PgBouncer?

#78
post #33

Earlier quoted context omitted.

Your high level buckets are languages but the constraints you list are usage patterns. You can build applications in any language that have many short-lived transactions in single connections or a few huge, blocking one, or anything in between. The former case is a good one for PGBouncer (it can interleave transactions) and the latter isn’t (it can’t do much about your three minute long BEGIN…COMMIT). Neither has to…

Languages have affordances. At the extreme ends are PHP, and Java or Go. PHP runs a separate logical process on every request which can't share resources with any other request. Almost everyone using Go is writing a long-running server process because that's how the libraries are designed. Almost everyone using Java is writing a long-running process or a module for one, because Java startup times are obscene. Java al…

Multi-threaded processes are used because that's the most efficient model, hardware wise. Tasks can maximally share resources. It's not to do with library design or startup time.

Re: Does anyone run Postgres without PgBouncer?

#79
post #41

Article title is pure click-bait. PgBouncer adds complexity. If you need it, you need it. If you don't need it, then you added complexity for nothing. > since neither IBM nor Oracle is a service that any self-respecting person not part of an enterprise sales cycle would actually use Author needs a serious ego check. There are legitimate engineering reasons to pick IBM Cloud (like if you need to support Z mainframes,…

It's an ironic claim because the Oracle cloud has great support for the Oracle Database, which doesn't require "bouncers" or equivalent. Its support for server-side connection pooling, client side load balancing (SCAN) and horizontal scaling means it offers exactly what the author wants - a single URL that just magically works and scales to any amount of work or connections cheaply. https://docs.oracle.com/en/databas…

Oracle licensing audits scare off plenty of folks.

At a small SaaS we were put off by the difficulty getting OracleDB's dev edition installed and working at all. While MySQL (then independent) and Pg were bare bones in comparison, they were very quick to get started, covered what was needed, and no risk of price spikes or time consuming audits. When pain points were encountered with MySQL and Pg there were plenty of flexible options to add into the mix.

Later I saw a competitor being crushed by licensing for Informix when they didn't need more than ~10% of its features.

Re: Does anyone run Postgres without PgBouncer?

#80

(I work on the postgres proxy layer at Neon) PgBouncer is entirely optional and it's not always the right choice. If you have a classical app (non serverless) and you can maintain a connection pool from your app, then I recommend avoiding pgbouncer. The benefits of pgbouncer mostly come from irregular client connections (too many, too much churn). If you don't have that problem, go direct to postgres. I'm exploring r…

Also some people with use cases that shouldn't need PgBouncer end up thinking they do because something else is misconfigured. This was brought up in a parent of https://news.ycombinator.com/item?id=49019695 ; FastAPI recommends dep-injecting transactions into your HTTP handlers. Ties up all your connections and creates idle xact spam. There are valid reasons to use PgBouncer, this isn't one. Even on serverless platf…

That’s ok if the number of workers is bounded and small. But if you’re operating on the order of hundreds to thousands of workers, maybe not so much.
Post reply on HN