Postgres rewritten in Rust, now passing 100% of the Postgres regression tests • Hacker News Board

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#201

Post by iigijshaba » Thu, Jul 09, 2026, 11:06 AM UTC

malisper wrote: post #160

queoahfh wrote: post #62

Earlier quoted context omitted.

From what I skimmed manually, not that many, but the code itself seems labyrinthical. Like, why have both Rust Try-supporting Error-like tagged union, but also booleans, for error handling, in the same function? https://github.com/malisper/pgrust/blob/3646a73515a5e4ac7d0b... https://github.com/malisper/pgrust/blob/3646a73515a5e4ac7d0b...

I'm not sure what you mean? The rust code you're showing mimics the Postgres code: https://github.com/postgres/postgres/blob/2e6578292a9184dcaa... The boolean being returned is the return value of the function. It's not used to return an error.

Sorry, I wrongly assumed in the C code when I skimmed it that the boolean was for error handling, not the result value. The elog() macro is used for error handling.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#202

Post by malisper » Thu, Jul 09, 2026, 11:07 AM UTC

boomskats wrote: post #188

malisper wrote: post #147

Hey author here. Wasn't expecting to see this up. To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture. I'm now working on a new, not yet published version of pgrust that inc…

This is great! Those analytical workloads numbers are mad - I'd love to see the benches, and I'm happy to contribute to some of the profiling. How does your thread-per-connection model compare to Heikki's proposal[0][1] from back in 2023? [0]: https://www.postgresql.org/message-id/31cc6df9-53fe-3cd9-af5... [1]: https://www.youtube.com/watch?v=xLLakMmVtbY

Rust actually made the change pretty simple. The main changes are:

  - Use thread local variables
  - Move everything from shared memory to process memory
  - Use threads instead of processes

I've started to see meaningful benefits by changing the parallel algorithms to use a shared memory space. For example parallel hash joins have to copy tuples through shared memory to pass them between workers. That's just not something I have to do.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#203

Post by Chyzwar » Thu, Jul 09, 2026, 11:12 AM UTC

I think the best way to test this would be to put PgBouncer or a similar proxy in front of a busy production database, and mirror queries to both traditional Postgres and the Rust one at the same time. Then you can compare output and performance under real load. After running it for a while, you could diff the tables one to one against the normal Postgres instance.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#204

Post by malisper » Thu, Jul 09, 2026, 11:23 AM UTC

tudorg wrote: post #172

malisper wrote: post #147

Hey author here. Wasn't expecting to see this up. To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture. I'm now working on a new, not yet published version of pgrust that inc…

> - Is ~300x faster than Postgres on analytical workloads. Right now it's 2x slower than Clickhouse on clickbench and I think it's possible to get faster than Clickhouse That sounds like you are storing the data in a columnar format? Or do you do both row and columnar? In a somewhat similar (yet also quite different) effort, I've been working on δx, a Postgres extension that compresses the data in a columnar format s…

Yep! The new version of pgrust supports batch based execution and a columnar format. I'm curious how you got δx to perform that well? From what I've seen a columnar layout only gets you part of the way and really good parallelism and really fast hash tables seem to make up a significant portion of why Clickhouse is faster.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#205

Post by OtomotO » Thu, Jul 09, 2026, 11:23 AM UTC

nextaccountic wrote: post #156

OtomotO wrote: post #92

Earlier quoted context omitted.

> Then, by giving them context or by post-training, you can make them sample non-average parts of the distribution they learned. How do you derive that something is "below average" or "average" or "above average"?

> How do you derive that something is "below average" or "average" or "above average"? One technique is RLHF: have an human expert assess it.

Mhm, I just wonder how many samples they get and how much time they have to come to the conclusion.

Like a short example is easier to grade, but not in the same ballpark as a whole codebase.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#206

Post by lukasco » Thu, Jul 09, 2026, 11:25 AM UTC

Quite a lot of projects are trying this "rewrite to a new language using LLM", both internally, or externally (like is here). For me, they confirm some (slightly controversial) takes.

1. human code reviews are dead. We don't yet know what's next. Two reasons they are dead: too much code to review, and code reviewing sucks (who wants to spend their days reviewing code?) 2. Not knowing how to review LLM code is a big barrier to adoption, but bigger regression test suites (testability/evals) is almost certainly the direction. 3. There are a lot of projects that haven't moved to more modern infra because it was too hard. Now it's much easier. Sure stuff will go wrong. Sure it all has to be tested. What's new here? 4. Programming languages for LLMs are coming. 5. Projects that don't allow AI coding will be forced to come around or fade.

Separately, bit off topic:

New projects will often have LLMs built in, so non-determinism will be inherent in the project. No amount of code review will be able to eliminate that.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#207

Post by lukasco » Thu, Jul 09, 2026, 11:30 AM UTC

sshine wrote: post #14

gingersnap wrote: post #3

I start to see a lot of these re-writes that depend on tests to state that its working. But the things that make software like Postgres and SQLite reliable are not mostly the test, but the real world production scars. That's where the reliability comes from, years and years of running in production.

> not mostly the test, but the real world production scars Most extensive test suites are exactly production scars: every time you have a bug or a regression, you write a test that confirms correct behaviour. SQLite is a good example to bring up because its extensive closed-source tests are what’s often cited as being what keeps people from forking it. (Turso did it, though, but it takes a company to deliver some gua…

So many comments here talking about the downsides. The only reason to do a rewrite is because there are massive upsides. Maybe the implicit point is that the upside (memory safety must be the biggest), isn't worth the downside (lots of bugs to be figured out before you trust it).

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#208

Post by iigijshaba » Thu, Jul 09, 2026, 11:42 AM UTC

Leynos wrote: post #154

queoahfh wrote: post #22

Earlier quoted context omitted.

Didn't the initial rewrite of Bun into Rust have an ocean of "unsafe" in it, and wasn't it entirely dysfunctional?

Yes, that was the point. It made unsafe behaviour visible in a way that could be addressed. I hadn't heard any reports of it being dysfunctional.

I have read up on it again, and while it was entirely dysfunctional at the very early stages, it quickly came up to par or beyond, with the LLM especially helped by the huge test suite written in Typescript, different from both Zig and Rust.

However, Jarred still describes a lot of unsafe, and usage of Miri in continuous integration.

Funnily enough, RAII is cited as a major benefit of rewriting from Zig to Rust, while C++ already has RAII. I wonder if C++ and Rust are more suited to larger programs than Zig, unless the architecture in Zig is handled carefully.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#209

Post by iigijshaba » Thu, Jul 09, 2026, 11:48 AM UTC

0x000xca0xfe wrote: post #185

mechazawa wrote: post #94

Earlier quoted context omitted.

Yeah same. The structure makes no real sense and when digging into the code it reads like I'm the first human to look at it.

I'm too young but I imagine assembly programmers were feeling the same when automatic code generation by compilers took over. Very weird.

More that I got confused by the C function returning bool, not as an error value, but as a result, which is my fault for skimming it quickly.

I have taken a closer look at the code, and it seems superficially a somewhat faithful rewrite, not quite idiomatic Rust, but closer than I anticipated at first. I know there are non-LLM rewriting tools for C to Rust, and with a test suite to help, a rewrite to Rust might be greatly helped. The new Rust code does have some drawbacks in some ways, and there are topics I am curious about.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#210

Post by bdelmas » Thu, Jul 09, 2026, 11:49 AM UTC

cryo32 wrote: post #123

dixtel wrote: post #64

Earlier quoted context omitted.

> software talibans I will note that, very funny

Yeah I'm using that one. We have a problem with software religious fundamentalists in our organisation and it's an apt description.

I actually had a lot of problems with software cult followers of influencer gurus like ThePrimeagen, Lex Fridman, Theo, etc... Those are so worst. You can't resonate with them.