There are so many in important things built that run perfectly fine on one postgres database, maybe redis as well and one to three of the smaller ec2 instances.
Postgres LISTEN/NOTIFY actually scales
71–80 of 99 posts
Re: Postgres LISTEN/NOTIFY actually scales
#72If I understnad right, they are saying it scales (to theri needs) with a custom patch to pg changing their semantics, right? It does seem interesting, and possibly welcome if there were a configuration option or even a way to set individual notifies as serialized or not.
To be clear, it's not a custom patch to pg itself, but an application-side buffering and batching optimization.
Ah, thanks. I am not following at all how an application can do this.
Re: Postgres LISTEN/NOTIFY actually scales
#73Earlier quoted context omitted.
The highest I could think of is WhatsApp, which gets ~1.6 million RPS on average, and they use MQTT
DynamoDB scales much more than that. In one of their dynamodb papers they claimed that the amazon us retail website alone made like 89 millions rps during prime day, a few years ago.
It's also one of the most aggravating database systems I've worked with.
Re: Postgres LISTEN/NOTIFY actually scales
#74Earlier quoted context omitted.
For reference, I've seen Visa marketing materials that suggest their network can do ~70k TPS. There are not very many systems one could conceive of that could do useful work 1/s for ~every human on the planet.
There is a tendency in tech to forget that there were highly scalable systems, processing huge numbers of transactions, before the internet and “hyper scalars” were a thing.
It's also worth noting that all of those systems could be served by your phone.
Re: Postgres LISTEN/NOTIFY actually scales
#75Earlier quoted context omitted.
Maybe there would be more if it were more straightforward to do so?
For reference, I've seen Visa marketing materials that suggest their network can do ~70k TPS. There are not very many systems one could conceive of that could do useful work 1/s for ~every human on the planet.
Hm, what am I not getting? One transaction per second for every human on the planet would be 8 billion TPS, not 70k, no?
Re: Postgres LISTEN/NOTIFY actually scales
#76[deleted]
Re: Postgres LISTEN/NOTIFY actually scales
#77One way it explicitly doesn't scale (unless something has changed since I last checked and my quick search failed me) is the hard limit on 8000 bytes of data in a notification. If your notification doesn't make sense to exist as a row (where you can just give an ID), then that makes it hard to use. I had a web game and the events were transient descriptions of changes in state that didn't make sense to store in the d…
Could you not send a message that pointed to the state that changed in a non-row-specific way?
Re: Postgres LISTEN/NOTIFY actually scales
#78[flagged]
Re: Postgres LISTEN/NOTIFY actually scales
#79Something interesting is that postgres PostGIS index for location finding scaled well enough for Uber. They eventually switched to something else but I don't think it was due to scaling problems.
Re: Postgres LISTEN/NOTIFY actually scales
#80I once was the CTO of a company that serviced about 100k requests per day across all our services. We grew to millions and eventually 10's of millions, but, somewhere along the way, an engineer on our team decided he wanted to build a queue off LISTEN/NOTIFY semantics in order to take advantage of strong consistency with the rest of our data model, which seemed reasonable given LISTEN/NOTIFY is not that hard to under…
I feel this is a management problem, and I disagree with the takeaway about adding a piece to the infra stack. "My devs don't want to work on part of our stack" is not a valid reason to add a new network node imo. Just make them work on it.