My recommendation to everyone out there willing to put in the effort of writing an article; don't put a naff low effort AI generated image at the top of it (or anywhere in it..). It turns me off before I've even read the first word.
Postgres LISTEN/NOTIFY actually scales
61–70 of 99 posts
Re: Postgres LISTEN/NOTIFY actually scales
#62Earlier quoted context omitted.
> 5 orders of magnitude too small for another. Nitpick on an otherwise good post, but I don’t think there are very many 6billion RPS systems out there, and those that do exist are almost certainly using bespoke, purpose-built tools
The highest I could think of is WhatsApp, which gets ~1.6 million RPS on average, and they use MQTT
[deleted]
Re: Postgres LISTEN/NOTIFY actually scales
#63Just sharing a data point and experience. We had a lot of success with LISTEN/NOTIFY when we paired it with a Rust graphql subscription broker. 10s of thousands of subscriptions, but only 3 or 4 LISTEN connections (one for each host). All changes would be pushed out to all hosts, who would each manage the actual user subscriptions and choose what to actually publish. This worked super well. In general, moving from hu…
I‘m interested, which graphql library for Rust can you recommend?
Re: Postgres LISTEN/NOTIFY actually scales
#64"Scale" isn't a binary, it's a continuum. "Scales to 60K/s" can be 5 orders of magnitude more than one system needs and 5 orders of magnitude too small for another. Personally I'd knock the general "premature optimization" off the list of "most common developer errors" and put in its place "using techs with the wrong scaling factors". If you use something too small and you exceed its needs, the failure is obvious, bu…
In this case it does by hitting hardware maximum. It scales until the max it is possible, due the bottleneck being in the hardware, not in the database or i/o. Check the article again:
> At maximum throughput, Postgres CPU is fully utilized, showing the database is actually saturated instead of bottlenecked on contention.
Re: Postgres LISTEN/NOTIFY actually scales
#65That makes one wonder, why is there only an option for globally ordered NOTIFYs?
I would expect to have a little more tuning knobs available:
- explicitly unordered
- ordered per table, but not globally
Re: Postgres LISTEN/NOTIFY actually scales
#66"Scale" isn't a binary, it's a continuum. "Scales to 60K/s" can be 5 orders of magnitude more than one system needs and 5 orders of magnitude too small for another. Personally I'd knock the general "premature optimization" off the list of "most common developer errors" and put in its place "using techs with the wrong scaling factors". If you use something too small and you exceed its needs, the failure is obvious, bu…
In this case it does by hitting hardware maximum. It scales until the max it is possible, due the bottleneck being in the hardware, not in the database or i/o. Check the article again: > At maximum throughput, Postgres CPU is fully utilized, showing the database is actually saturated instead of bottlenecked on contention.
It doesn’t necessarily follow that because Postgres is “fully utilised” it is hitting “hardware maximum”, or even that it’s not bottlenecked on contention.
As an example, spinlocks can push CPU usage very high whilst being an obvious symptom of contention.
Re: Postgres LISTEN/NOTIFY actually scales
#67Earlier quoted context omitted.
Not saying this in a mean way: It seems to me this can be boiled down to "using things without understanding how they work doesn't scale". Yes, vanilla listen/notify doesn't scale. But the OP actually figured out how to make it scale. So your engineers don't have to. As the community builds, over time, distributed systems, it also understands which brick can do what, and it turns out that starting with less bricks an…
Nah this isn't mean at all. I think this is the correct takeaway. I actually felt like I understood how the system worked and it wasn't that difficult for me, but I also understood how, as we were adding engineers who were all very under water, the idea of learning our queueing system to modify a core feature it powered seemed like a big mental context switch (bigger than it actually was). I noted this in another com…
Also the results given it the article are still worse than redis with zero work and no risk of breaking later, both in nb of req and latency.
And you can move redis to another server next later if needed to split the db and notification load.
Re: Postgres LISTEN/NOTIFY actually scales
#68Earlier quoted context omitted.
Maybe there would be more if it were more straightforward to do so?
For reference, I've seen Visa marketing materials that suggest their network can do ~70k TPS. There are not very many systems one could conceive of that could do useful work 1/s for ~every human on the planet.
There is a tendency in tech to forget that there were highly scalable systems, processing huge numbers of transactions, before the internet and “hyper scalars” were a thing.
Re: Postgres LISTEN/NOTIFY actually scales
#69[flagged]
Re: Postgres LISTEN/NOTIFY actually scales
#70Much of the FUD around using LISTEN/NOTIFY in production comes from people who have never done so. It of course has its limits, and we should remain aware of them. Much like the limits of every piece of tech in our stacks. Choose database queue technology https://news.ycombinator.com/item?id=37636841
Personally I love it for cache invalidation. I used it on an older especially on project originally built without caching in mind that then added caching for immutable data without caching for mutable data in mind. It fell to me to add caching of mutable data. I found it rather convenient to put NOTIFY triggers on cached tables and have a LISTENer that will delete the corresponding rows from memcached when it gets a…
and here was I thinking ( when creating same cache invalidation system ) that I was the only one that came up with this patch. LMAO