Live data from Hacker News

An early look at Postgres 14: Performance and monitoring Improvements

pganalyze.com

111–120 of 254 posts

Re: An early look at Postgres 14: Performance and monitoring Improvements

#111
It would be nice to hear how much of problem XID wraparound is in Postgres 14 - do the fixes below address it entirely or just make it less of a problem?

I see no mention of addressing transaction id wraparound, but these are in the release notes:

Cause vacuum operations to be aggressive if the table is near xid or multixact wraparound (Masahiko Sawada, Peter Geoghegan)

This is controlled by vacuum_failsafe_age and vacuum_multixact_failsafe_age.

Increase warning time and hard limit before transaction id and multi-transaction wraparound (Noah Misch)

This should reduce the possibility of failures that occur without having issued warnings about wraparound.

https://www.postgresql.org/docs/14/release-14.html

Re: An early look at Postgres 14: Performance and monitoring Improvements

#112

Postgres is one of those pieces of software that’s so much better than anything else, it’s really incredible. I wonder if it’s even possible for competitors to catch up at this point - there’s not a lot of room for improvement in architecture of relational databases any more. I’m starting to think that Postgres is going to be with us for decades maybe even centuries. Do any other entrenched software projects come to…

I'm an enormous fan of Postgres, it's my default go-to RDBMS. But the memory expense of connections is a huge issue and this article doesn't convince me that it's solved. The machine being used for this benchmark has 96 vCPUs, 192G of RAM, and costs $3k/mo. My business runs just fine on a 3.75G, 1 vCPU instance. But idle connections eat up a huge amount of RAM and I sometimes find myself hitting the limits when a loa…

Out of curiosity, do you know what causes this?

Re: An early look at Postgres 14: Performance and monitoring Improvements

#113
post #75

Earlier quoted context omitted.

> Do any other entrenched software projects come to mind? Elasticsearch is underrated here, IMO. Yes, there are alternatives for simple fulltext search. But there’s a lot more it can do (adhoc aggregations incorporating complex fulltext searches, with custom scripted components; geospatial; index lifecycle management) and if you’re using those features, there’s nothing else comparable. It’s pretty stable, too, once y…

I don't know about elasticsearch specifically, but I'm skeptical of special-purpose systems for databases. They are great in some cases and terrible in others, and over time, use cases push database systems into their worst cases. Use cases rarely stay in the sweet spot of a special-purpose system. That being said, if the integration is great, and/or the special system is a secondary one (fed from a general-purpose s…

I’m not sure I fully understand your comment (databases that are special-purpose and evolve out of a sweet spot, or special-purpose systems using databases in worst-case ways?).

I certainly wouldn’t say ES is the former. We use it for some conplex things that (AFAIK) no other (publicly available; I don’t what eg Twitter or Google has going on) system could provide at the scale we need. Everything we’re doing is well within the realm of what ES is built for, and it’s the only system built for it. It’s not perfect, but most of our performance issues could be solved by scaling out, where query or index optimization isn’t tractable.

Re: An early look at Postgres 14: Performance and monitoring Improvements

#114
post #99
post #88

Earlier quoted context omitted.

Wow is this for real? That is such a big quality of life change! Happy to see it!

Not much different from some_jsonb#>>'{some,path}' and once you add the need to convert out of jsonb to text, you'll not be saving any characters either. At least for queries. For updates, it looks nice I guess.

I think the difference is familiarity.

It shouldn't matter so much, but when you don't use one language as much as you do other languages, it becomes that much harder to remember unfamiliar syntaxes and grammars, and easier to confuse similar looking operations with each other.

Re: An early look at Postgres 14: Performance and monitoring Improvements

#115

Any progress on high availability deployments yet? Or does it still rely on problematic, 3rd party tools? Last time I was responsible for setting up a HA Postgres cluster it was a garbage fire, but that was nearly 10 years ago now. I ask every so often to see if it has improved and each time, so far, the answer has been no.

If you want HA use AWS RDS, Azure Citus, GCP Cloud SQL. Otherwise use MySQL, Oracle, MongoDB, Cassandra etc if you want to run it on your own. Any other database that invested in a native and supported HA/clustering implementation.

Cockroachdb or Yugabyte work well for some cases you might use postgres for.

Re: An early look at Postgres 14: Performance and monitoring Improvements

#116

Postgres is one of those pieces of software that’s so much better than anything else, it’s really incredible. I wonder if it’s even possible for competitors to catch up at this point - there’s not a lot of room for improvement in architecture of relational databases any more. I’m starting to think that Postgres is going to be with us for decades maybe even centuries. Do any other entrenched software projects come to…

There's a ton of room for improvement in the architecture of relational databases. This isn't a dig against Postgres, or ignoring how difficult it will be to get a new system to the same level of maturity. But databases designed natively for cloud/clustering, SSDs, (pmem soon perhaps), etc are quite a bit different. There's enormous simplifications and performance gains possible. There's been a lot of exciting work i…

Yeah, to list a bit:

- scaling is non-trivial (you can't just add a node and have PostgreSQL automagically Do The Right Thing™)

- you can only have so many connections open to the database, causing issues with things such as AWS Lambda

- I don't remember if this was changed, but I got the impression a while ago that having dynamic DB users was a bit cumbersome to set up (plug PostgreSQL to AD/LDAP)

Re: An early look at Postgres 14: Performance and monitoring Improvements

#117

Earlier quoted context omitted.

Cockroach is the worst brand for a database ever. Even Croach would be a massive branding improvement. This is similar to how gimp is a terrible brand.

I mean... the WORST? For me Mongo takes the cake, but oracle is up there too.

Really? Oracle actually makes a lot of sense to me for a database name (in the 'source of truth' sense, not in the prophet sense).

Mongo, on the other hand, has definitely always had the racist/ablist slur as the first connotation for me.

Re: An early look at Postgres 14: Performance and monitoring Improvements

#118

Earlier quoted context omitted.

MySQL and mariadb have first class temporal tables. Pg has compile requirement and so cannot use in AWS RDS.

> MySQL and mariadb have first class temporal tables. Pg has compile requirement and so cannot use in AWS RDS. There’s a pl/pgsql reimplementation of temporal tables specifically for that use case.

https://news.ycombinator.com/item?id=26768220

Re: An early look at Postgres 14: Performance and monitoring Improvements

#119

It would be nice to hear how much of problem XID wraparound is in Postgres 14 - do the fixes below address it entirely or just make it less of a problem? I see no mention of addressing transaction id wraparound, but these are in the release notes: Cause vacuum operations to be aggressive if the table is near xid or multixact wraparound (Masahiko Sawada, Peter Geoghegan) This is controlled by vacuum_failsafe_age and v…

Co-author of that feature here.

Clearly it doesn't eliminate the possibility of wraparound failure entirely. Say for example you had a leaked replication slot that blocks cleanup by VACUUM for days or months. It'll also block freezing completely, and so a wraparound failure (where the system won't accept writes) becomes almost inevitable. This is a scenario where the failsafe mechanism won't make any difference at all, since it's just as inevitable (in the absence of DBA intervention).

A more interesting question is how much of a reduction in risk there is if you make certain modest assumptions about the running system, such as assuming that VACUUM can freeze the tuples that need to be frozen to avert wraparound. Then it becomes a question of VACUUM keeping up with the ongoing consumption of XIDs by the system -- the ability of VACUUM to freeze tuples and advance the relfrozenxid for the "oldest" table before XID consumption makes the relfrozenxid dangerously far in the past. It's very hard to model that and make any generalizations, but I believe in practice that the failsafe makes a huge difference, because it stops VACUUM from performing further index vacuuming.

In cases at real risk of wraparound failure, the risk tends to come from the variability in how long index vacuuming takes -- index vacuuming has a pretty non-linear cost, whereas all the other overheads are much more linear and therefore much more predictable. Having the ability to just drop those steps if and only if the situation visibly starts to get out of hand is therefore something I expect to be very useful in practice. Though it's hard to prove it.

Long term, the way to fix this is to come up with a design that doesn't need to freeze at all. But that's much harder.

Re: An early look at Postgres 14: Performance and monitoring Improvements

#120
post #113

Earlier quoted context omitted.

I don't know about elasticsearch specifically, but I'm skeptical of special-purpose systems for databases. They are great in some cases and terrible in others, and over time, use cases push database systems into their worst cases. Use cases rarely stay in the sweet spot of a special-purpose system. That being said, if the integration is great, and/or the special system is a secondary one (fed from a general-purpose s…

I’m not sure I fully understand your comment (databases that are special-purpose and evolve out of a sweet spot, or special-purpose systems using databases in worst-case ways?). I certainly wouldn’t say ES is the former. We use it for some conplex things that (AFAIK) no other (publicly available; I don’t what eg Twitter or Google has going on) system could provide at the scale we need. Everything we’re doing is well…

I interpreted (misinterpreted?) your comment to be suggesting ES for wider use cases.
Post reply on HN