Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

github.com

Post reply on HN
Print view View original

221–230 of 756 posts

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#221

Post by Apaec »

malisper wrote: post #202
boomskats wrote: post #188

Earlier quoted context omitted.

This is great! Those analytical workloads numbers are mad - I'd love to see the benches, and I'm happy to contribute to some of the profiling. How does your thread-per-connection model compare to Heikki's proposal[0][1] from back in 2023? [0]: https://www.postgresql.org/message-id/31cc6df9-53fe-3cd9-af5... [1]: https://www.youtube.com/watch?v=xLLakMmVtbY

Rust actually made the change pretty simple. The main changes are: - Use thread local variables - Move everything from shared memory to process memory - Use threads instead of processes I've started to see meaningful benefits by changing the parallel algorithms to use a shared memory space. For example parallel hash joins have to copy tuples through shared memory to pass them between workers. That's just not somethin…

[flagged]

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#222

Post by Suzuran »

gingersnap wrote: post #3

I start to see a lot of these re-writes that depend on tests to state that its working. But the things that make software like Postgres and SQLite reliable are not mostly the test, but the real world production scars. That's where the reliability comes from, years and years of running in production.

That's not relevant though. All concerns are secondary to security and Rust is the only language with security GUARANTEES. No other language is as secure. Therefore, even the worst Rust rewrite is automatically better than the best work in any other language, because it is the only one with guaranteed security.

If a Rust rewrite of any of your software becomes available and you aren't installing it immediately and without reservation, then you are simply not giving security the priority it both demands and deserves, and that makes you disastrously insecure. This is a serious issue that should be given all priority. There is no room for debate. Your only policies should be security before all else and compliance with those policies must be absolute and without deviation, or all is lost.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#223

Post by vitally3643 »

kelnos wrote: post #34
sshine wrote: post #14

Earlier quoted context omitted.

> not mostly the test, but the real world production scars Most extensive test suites are exactly production scars: every time you have a bug or a regression, you write a test that confirms correct behaviour. SQLite is a good example to bring up because its extensive closed-source tests are what’s often cited as being what keeps people from forking it. (Turso did it, though, but it takes a company to deliver some gua…

Sure, but behaviors that never have a bug or regression don't get a test. Software of this kind of complexity has all kinds of behavior that has never been broken, and doesn't have a specific test written for it. Getting an extensive test suite passing is certainly orders of magnitude better than having no test suite at all, but it still doesn't tell you as much as you need to know. I would absolutely never trust an…

If that's your concern, then your argument becomes "software should never change". Why dare patch any bug ever? It might be load-bearing in some unknown, undocumented, unsupported workflow somewhere in the world. No test imaginable can catch that apart from the scream test.

There are reasonable arguments against language ports, but this is not one. You're making an argument against code changing at all ever.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#224

Post by kjs3 »

kstrauser wrote: post #7
gingersnap wrote: post #3

I start to see a lot of these re-writes that depend on tests to state that its working. But the things that make software like Postgres and SQLite reliable are not mostly the test, but the real world production scars. That's where the reliability comes from, years and years of running in production.

In a project like PostgreSQL, those scars are reflected in unit tests demonstrating that they’re fixed. It’d be hard to pass its test suite and not be as robust as the original.

I dunno...I can envision something vibecoded prioritizing passing test suites producing something that does that, but isn't even functional in real-world production. Sort of like in the pre-AI world, where someone claims 'standards compliance' by way of passing compliance test suites, but can't actually interoperate well with other implementations of the standard. YMMV.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#225

Post by pknerd »

ZiiS wrote: post #26

What would be interesting is if they found a memory unsafe bug. Postgres is a perfect case study of 30 years of C with a bit of CPP; if rewriting in a safer language didn't find anything...

> What would be interesting is if they found a memory unsafe bug

They will ask relevant Claude skill.md

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#226

Post by surajrmal »

kstrauser wrote: post #7
gingersnap wrote: post #3

I start to see a lot of these re-writes that depend on tests to state that its working. But the things that make software like Postgres and SQLite reliable are not mostly the test, but the real world production scars. That's where the reliability comes from, years and years of running in production.

In a project like PostgreSQL, those scars are reflected in unit tests demonstrating that they’re fixed. It’d be hard to pass its test suite and not be as robust as the original.

Unit tests aren't useful for rewrites, only integration tests are. So there may be missing coverage. Also many things are simply difficult to test (eg performance under very specific conditions)

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#227

Post by dapperdrake »

gingersnap wrote: post #3

I start to see a lot of these re-writes that depend on tests to state that its working. But the things that make software like Postgres and SQLite reliable are not mostly the test, but the real world production scars. That's where the reliability comes from, years and years of running in production.

"Everybody has a production system. The lucky ones also have a test system."

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#228

Post by bbg2401 »

Suzuran wrote: post #222
gingersnap wrote: post #3

I start to see a lot of these re-writes that depend on tests to state that its working. But the things that make software like Postgres and SQLite reliable are not mostly the test, but the real world production scars. That's where the reliability comes from, years and years of running in production.

That's not relevant though. All concerns are secondary to security and Rust is the only language with security GUARANTEES. No other language is as secure. Therefore, even the worst Rust rewrite is automatically better than the best work in any other language, because it is the only one with guaranteed security. If a Rust rewrite of any of your software becomes available and you aren't installing it immediately and wi…

> If a Rust rewrite of any of your software becomes available and you aren't installing it immediately and without reservation

This is silly.

Rust is awesome, and it's hard to argue against in many domains. However, software is more than the language it is written in or the runtime serving it. Is the Rust rewrite fully compatible? Is it supported by a strong community? Is it likely to continue to be supported? Is its release cadence sensible? Is its licence compatible with your intended usage?

There are many questions needing to be answered before making rash decisions based purely on tech.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#229

Post by guenthert »

eu-tech-tak wrote: post #72

How is the performance compared to regular PostgreSQL? I know it says it is not performance optimized yet, but if this succeeds, will it only bring more "memory safety" or is there a serious performance gain as well?

This seems to be a multi-phased project. First phase (completed) was the re-write in Rust. There doesn't seem to be a performance gain and no significant one should be expected. In a 2nd phase a new architecture is implemented which malisper claims to perform much better.

I wished the two phases would have been tackled in reverse order.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#230

Post by su66u »

dirkc wrote: post #78

How would one go about reviewing a piece of code like this? One of the things I'd typically do is peek at the commit history. Seeing what people worked on and how they did it tends to say a lot about a project. But with LLMs generating 7101 commits in less than a month that isn't feasible. Even looking at a single day is way too much [1]. It probably also doesn't make sense since the commits content won't tell you mu…

[dead]
Post reply on HN