Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Does anyone run Postgres without PgBouncer?

brandur.org

Post reply on HN
Print view View original

61–70 of 121 posts

Re: Does anyone run Postgres without PgBouncer?

#61

Post by moooo99 »

mike_hearn wrote: post #55
solatic wrote: post #41

Article title is pure click-bait. PgBouncer adds complexity. If you need it, you need it. If you don't need it, then you added complexity for nothing. > since neither IBM nor Oracle is a service that any self-respecting person not part of an enterprise sales cycle would actually use Author needs a serious ego check. There are legitimate engineering reasons to pick IBM Cloud (like if you need to support Z mainframes,…

It's an ironic claim because the Oracle cloud has great support for the Oracle Database, which doesn't require "bouncers" or equivalent. Its support for server-side connection pooling, client side load balancing (SCAN) and horizontal scaling means it offers exactly what the author wants - a single URL that just magically works and scales to any amount of work or connections cheaply. https://docs.oracle.com/en/databas…

I do not think it is a mystery why people favor Postgres over OracleDB, is it?

Re: Does anyone run Postgres without PgBouncer?

#62

Post by btown »

mike_hearn wrote: post #55
solatic wrote: post #41

Article title is pure click-bait. PgBouncer adds complexity. If you need it, you need it. If you don't need it, then you added complexity for nothing. > since neither IBM nor Oracle is a service that any self-respecting person not part of an enterprise sales cycle would actually use Author needs a serious ego check. There are legitimate engineering reasons to pick IBM Cloud (like if you need to support Z mainframes,…

It's an ironic claim because the Oracle cloud has great support for the Oracle Database, which doesn't require "bouncers" or equivalent. Its support for server-side connection pooling, client side load balancing (SCAN) and horizontal scaling means it offers exactly what the author wants - a single URL that just magically works and scales to any amount of work or connections cheaply. https://docs.oracle.com/en/databas…

Slightly off topic, but a friend of mine worked on Microsoft SQL Server a ~decade ago, and I recall all sorts of conversations on types of optimizations they were doing that sounded truly magical compared to other databases at the time. There was a meaningful frustration on the team that their features weren't getting as much hype as the databases du jour.

I think it's very relevant to consider that "enterprise" adjacent databases may not support the licensing you need at scale - but that doesn't mean they don't have great engineering and research teams that are solving really difficult challenges, and those challenges may be highly relevant to your workflows. Go into things with an open mind, if not an open wallet!

Re: Does anyone run Postgres without PgBouncer?

#64

Post by aobdev »

SoftTalker wrote: post #54
aobdev wrote: post #53

Earlier quoted context omitted.

This solves the latency but not the overhead--you'll end up with a full pool of connections for each running instance of your application.

If I'm implementing my own connection pool, I'll certainly make the number of connections configurable.

That's fine but doesn't address the issue that PgBouncer does. Your application connection pool can multiplex all the connections needed in one application. PgBouncer can multiplex the connections across all applications (whether different apps or many instances of the same app).

Re: Does anyone run Postgres without PgBouncer?

#65

Post by stackskipton »

atombender wrote: post #57
fabian2k wrote: post #34

If you have a connection pooler in your application, and the DB is only used for this application, you don't need an external pool like PgBouncer. That is probably a pretty common scenario, and typical web frameworks include a connection pool anyway. An additional complication is that more aggressive pooling methods have side effects that you must know and prevent in your application. They're not safe to use out of t…

It's a little more nuanced than that. At my company we run Kubernetes workloads where we have 20-30 or sometimes more pods connecting to a single Postgres instance, each pod handling hundreds of concurrent requests. If each pod hangs on to pooled connections for more than a few seconds, then you end up with quite a few Postgres processes and a lot of memory usage and process churn; we mitigate that by having a relati…

SRE here, that's a ton of pods and sounds like it's language or architecture if you are serving up so few requests per pod.

We have Kubernetes here too for Java monoliths and we have 8 Pods serving more than that. Since Java has connection pooling, the overhead has never been enough to justify overhead of Pgbouncer for this Java app.

Re: Does anyone run Postgres without PgBouncer?

#66

Post by pocksuppet »

ummonk wrote: post #35
petcat wrote: post #23

This question will get more interesting responses if it was qualified as: "Does anyone run Postgres without PgBouncer for non-trivial workloads?" Because, as we can see from the comments so far, lots of people are going to say you don't need it for your blog that gets 10 hits a month. I've personally never heard of anyone not using PGbouncer, or some connection pooling proxy, for reasonably concurrent workloads. PG's…

I feel like there are a lot of use cases where I’d opt for SQLite and a lot of use cases where I’d opt for Postgres + PgBouncer. I’m curious what kinds of features push towards using Postgres alone over SQLite.

Concurrency, data types, scalability, centralization, or replication push for Postgres. The push against PgBouncer is that you don't need it, unless you do. If you have an app-level connection pool, you probably don't need PgBouncer.

Re: Does anyone run Postgres without PgBouncer?

#67

Post by pocksuppet »

tyre wrote: post #33
matsemann wrote: post #21

Python: absolutely necessary due to the amount of processes and various deployments to run an application once it grows. Java: never felt the need even on quite big apps. As it's much easier to share a connection pool locally, it's not as many single connections across the whole app.

Your high level buckets are languages but the constraints you list are usage patterns. You can build applications in any language that have many short-lived transactions in single connections or a few huge, blocking one, or anything in between. The former case is a good one for PGBouncer (it can interleave transactions) and the latter isn’t (it can’t do much about your three minute long BEGIN…COMMIT). Neither has to…

Languages have affordances. At the extreme ends are PHP, and Java or Go. PHP runs a separate logical process on every request which can't share resources with any other request. Almost everyone using Go is writing a long-running server process because that's how the libraries are designed. Almost everyone using Java is writing a long-running process or a module for one, because Java startup times are obscene. Java also has a very convenient synchronization primitive.

Re: Does anyone run Postgres without PgBouncer?

#68

Post by mrighele »

petcat wrote: post #23

This question will get more interesting responses if it was qualified as: "Does anyone run Postgres without PgBouncer for non-trivial workloads?" Because, as we can see from the comments so far, lots of people are going to say you don't need it for your blog that gets 10 hits a month. I've personally never heard of anyone not using PGbouncer, or some connection pooling proxy, for reasonably concurrent workloads. PG's…

Not all non-trivial workloads are web-scale. There are plenty of on-premise applications out there that have at most hundreds or thousands of concurrent users, and the connections come from a bunch of fat spring boot servers that handle most of the pooling by themselves.

Re: Does anyone run Postgres without PgBouncer?

#69

Post by KronisLV »

> since neither IBM nor Oracle is a service that any self-respecting person not part of an enterprise sales cycle would actually use

Heh, snarky.

I guess (almost) everyone uses PgBouncer because they want to use a setup that will need the scaling needs of most users from the get-go, to avoid wasting support time.

Personally, I run without it due to a fairly small scale - I don't need more than about 128-256 connections max and for me a dedicated connection pooler (other than what's sometimes used app-side) would just add complexity.

At the same time, one could totally reasonably make the argument that if almost everyone uses it, then it SHOULD quite possibly be a built in feature, instead of a separate component - such a tighter integration would most likely bring the overall complexity down.

Re: Does anyone run Postgres without PgBouncer?

#70

Post by ndriscoll »

aobdev wrote: post #53
gkbrk wrote: post #31

Earlier quoted context omitted.

If it's designed well, your application also opens a pool of connections and re-uses them each time the code needs a DB connection.

This solves the latency but not the overhead--you'll end up with a full pool of connections for each running instance of your application.

How many running instances do you need though? e.g. Scala web frameworks should be able to do thousands of RPS on a single core without the application developer really trying to optimize anything, and I always hear that even Ruby, Python, etc. are also fast enough to be IO bound so you should just need 2 copies for redundancy, right? Then give each like 8-16 connections.
Post reply on HN