Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Important PostgreSQL 14 update to avoid silent corruption of indexes

migops.com

Post reply on HN
Print view View original

91–100 of 101 posts

Re: Important PostgreSQL 14 update to avoid silent corruption of indexes

#91

Post by urthor »

bigiain wrote: post #39
SonOfLilit wrote: post #17

Earlier quoted context omitted.

Large C codebases _have_ to be exceptionally nice, or they immediately collapse under their own weight. As a dev team, the language teaches you this the hard way. I've never seen a terrible huge C codebase (but have seen many in other languages).

> I've never seen a terrible huge C codebase I have 100% confidence they exist. They just don't get uploaded to Github out of shame or embarrasment.

I think OP just found a really well done C file.

You can definitely see many famous projects on GitHub, where even though the code is logical, you can't help feel it's a bit all over the place.

cpython isn't all sun and roses.

A broad generalization, but with C and C++ I'd say the standards for "good code" have dramatically risen in the last 10 years.

Variable names seem to have doubled in size.

Re: Important PostgreSQL 14 update to avoid silent corruption of indexes

#93

Post by x4m »

mattashii wrote: post #84
ants_a wrote: post #82

Earlier quoted context omitted.

Simpler fix is to upgrade to fixed version and reindex concurrently any indexes that could have been affected.

That fixed version is not yet released (that is planned for 2022-06-16, in 6 days), and database upgrades are not trivial. Sure, it's easier to "just go to the version that has this fixed", but if that's not possible, this might just be the next best thing.

Just revert d9d0762 or use actual REL_14_STABLE.

Re: Important PostgreSQL 14 update to avoid silent corruption of indexes

#96

Post by Someone »

throwawaydingus wrote: post #19
mattashii wrote: post #5

It is noted in the post, I'll repeat it to be clear: This corruption can only occur _during_ a (re)index with CONCURRENTLY specified, on rows that are modified during the reindex operation, and only for that index. No other indexes are impacted, and an index can only be impacted when the updates on the table don't update indexed columns. Nevertheless, if you frequently run CIC, you could be having this issue -- right…

What? That seems absurdly apologist even for me and I love postgres. Everyone seems to be downplaying this corruption issue saying it only happens when x or y and it can be avoided by doing complex operation z but… if build anything that isn’t a toy or demo on Postgres you NEED to build indexes concurrently. They shipped optimization to a feature to let indexes be built concurrently yet failed to perform thorough eno…

> They shipped optimization to a feature to let indexes be built concurrently yet failed to perform thorough enough testing to ensure concurrent updates don’t disrupt the correctness of the index?

From what I can tell, a problem _may_ occur in those circumstances, but there’s no guarantee it _will_ occur. That can make finding it in testing difficult.

A performant ACID database is a very complex piece of software. It is a pipe dream to think about any such feature would ship without bugs.

See for example https://support.microsoft.com/en-us/topic/fix-data-corruptio..., which shows a similar issue on SQL Server.

Re: Important PostgreSQL 14 update to avoid silent corruption of indexes

#97

Post by e12e »

bigiain wrote: post #39
SonOfLilit wrote: post #17

Earlier quoted context omitted.

Large C codebases _have_ to be exceptionally nice, or they immediately collapse under their own weight. As a dev team, the language teaches you this the hard way. I've never seen a terrible huge C codebase (but have seen many in other languages).

> I've never seen a terrible huge C codebase I have 100% confidence they exist. They just don't get uploaded to Github out of shame or embarrasment.

> I've never seen a terrible huge C codebase

libssl/openssl (before the rewrite/fixes especially)? Maybe doesn't qualify as huge?

See eg: https://www.youtube.com/watch?v=GnBbhXBDmwU "LibreSSL with Bob Beck" (the first 30 days)

Re: Important PostgreSQL 14 update to avoid silent corruption of indexes

#98

Post by Tostino »

mattashii wrote: post #84
ants_a wrote: post #82

Earlier quoted context omitted.

Simpler fix is to upgrade to fixed version and reindex concurrently any indexes that could have been affected.

That fixed version is not yet released (that is planned for 2022-06-16, in 6 days), and database upgrades are not trivial. Sure, it's easier to "just go to the version that has this fixed", but if that's not possible, this might just be the next best thing.

Minor version upgrades are trivial, just a binary swap and restart away.

Re: Important PostgreSQL 14 update to avoid silent corruption of indexes

#99

Post by jvolkman »

cperciva wrote: post #58
jvolkman wrote: post #13

Earlier quoted context omitted.

I get the impression that long files are culturally acceptable in systems-level C code. E.g., just a cherry-picked file from Linux: kernel/sched/core.c is over 11k lines. https://github.com/torvalds/linux/blob/master/kernel/sched/c...

There are 18609 .c files in my checked-out copy of the FreeBSD src tree. The median length is 258 lines; 90% are 1373 lines or shorter; 99% are 5241 lines or shorter. The statistics for the 6071 .c files in the FreeBSD kernel are somewhat higher -- median is 460 lines; 90th percentile is 2070 lines; 99th percentile is 7678 lines -- but your example of a 11133 line file is definitely at the extreme high end.

I didn't run any stats when I found that file. Just clicked around in github a handful of times looking for something that seemed like it'd be complex.

Re: Important PostgreSQL 14 update to avoid silent corruption of indexes

#100

Post by vfclists »

TedShiller wrote: post #20

I love Postgres. But when I look at the bug list of every release it makes me scared. The types of bugs they have are indicative of a poor development process.

Surprising how people feel the need to respond to a post by someone with the handle "TedShiller".

Is that a real name?

Sorry if it is and I hurt your feelings.

Post reply on HN