Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

github.com

Post reply on HN
Print view View original

731–740 of 756 posts

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#731

Post by ceejayoz »

SirHackalot wrote: post #689
addedGone wrote: post #635

Earlier quoted context omitted.

In the future, code readability will not really matter, the concept of maintainability will also change: Is it maintainable by AI, if yes, then it's maintainable, we will approach the day where we have virtually ZERO code written by human, so all the tools must be built for AIs, not humans anymore.

Perhaps in the future my grandmother will have wheels… But today radiologists still do radiology (despite AI being better at reading scans), taxi drivers still have work, and truck drivers are still able to put food on the table. It’s still 2026, time traveler, so touch grass…

Radiology is using AI extensively already. I was at a radiology conference in Boston recently; entire session tracks on it. Apparently 80% of FDA-approved AI medical devices are in radiology.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#732

Post by queenkjuul »

Petersipoi wrote: post #369
queenkjuul wrote: post #314

Earlier quoted context omitted.

It's the notably poor quality of LLM-generated code

As opposed to the incredible code that humans are known to write...

I mean, yes? Every engineer on my team produces better code than Claude. There are plenty of examples of excellent human produced code, so this argument always falls flat for me.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#733

Post by baranul »

NohatCoder wrote: post #702
ramijames wrote: post #651

Earlier quoted context omitted.

I'm curious. Do you attribute this to weak and/or incomplete tests? How granular should tests be to have complete coverage so that an AI won't create a converted codebase that "passes tests" but is still functionally inaccurate?

There is no such thing as a complete test suite, there will always be some possible bug that it doesn't catch. In particular, if you put an LLM in an automated loop of "this test fails, please fix it", there is a pretty good chance that it will simply special case all of the tests, possibly in some contrived way that makes it not at all obvious when you read the code.

This is what I have observed as well. Been through many debates about the "perfect plan" and "perfect tests" fallacies.

Maybe a way of looking at it, to understand the nature of the issue. Have a LLM translate a novel from English to Spanish. Of course it can do that translation at speeds that no human could (score a point for AI). But how good is the Spanish translation? Is the quality better than what humans could do? Wouldn't those who are not fluent in Spanish be more easily impressed?

We then can do all kinds of configuration setups and tests, but how do we know the Spanish was translated perfectly, without a massive detail review (and being already truly bilingual in both English and Spanish)?

As is the usual case in the pursuit of perfection (which nothing in nature ever seems to be), there is going to be mistakes, costs (worth it?), and gray areas. It would be foolhardy for us not to suspect or pass it off as otherwise.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#734

Post by baranul »

ismailmaj wrote: post #441
malisper wrote: post #270

Earlier quoted context omitted.

My approach has changed throughout the course of this project. Throughout most of the project, we were working off of a c2rust translation of Postgres to Rust. That gave us a bunch of Rust code that was unsafe but did pass the Postgres test suite and was fast. c2rust had split Postgres into 1000 different crates. We then went through 1 by 1 and rewrote each crate into idiomatic rust. This naturally lended itself to a…

I think I would be horrified looking at your Claude API bill.

Yeah, like to see more disclosures about this. Not to mention the whole background debate on the real costs for the AI companies, at what point prices go up to reflect that, or the possible consequences of being overly reliant on this technology.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#735

Post by CamouflagedKiwi »

jiggawatts wrote: post #704
CamouflagedKiwi wrote: post #687

Earlier quoted context omitted.

Because you know that the current one works. If you have a bank running on COBOL (or whatever), you've had that for 30+ years now, so while it might have bugs, you know what they are. You don't know what the LLM output is. Hence back to my original point: writing the code is not the hard bit. Making yourself (and your CEO etc) comfortable to put that into production is one of the hard bits.

> Because you know that the current one works. What do you even mean by "works", specifically? > it might have bugs, you know what they are. Okay, so it doesn't work, you know it doesn't work, it's just that you accept the specific ways in which it doesn't work. I've lost track of all the myriad stupid ways in which these ancient systems are hugely ineffectual without even being outright faulty. Like airline tickets…

> What do you even mean by "works", specifically?

It runs and accepts people's payments, which means you're not on the front page of the newspaper (not in a good way).

> ... because they're stored in terrible ancient mainframe databases, the text fields all have tiny maximum lengths. Hence they're all abbreviations. Attacker-controlled abbreviations without any authenticity assurance of any kind!

They also have to go through payment networks which are very often the limits on those things. So yeah, it sucks, but just fixing one DB isn't enough - the whole thing has to get upgraded.

> I have no idea who actually transfers money out of my accounts! There are no URLs, no metadata, nothing to actually confirm the identity of the other party. Every field in a transaction record is 100% attacker-controlled and unverified by my bank.

There is a little bit, but not much. Again, if these transfers are happening via card, it's all a terrible old fixed-length setup. Would be great if it was better, but you need Visa and Mastercard to upgrade as well. And of course there _is_ verification - most banks don't do a great job of surfacing this but they know if they've verified a PIN or CVC, or if it was contactless (in which case it _is_ unverified, but society realised we prefer the convenience there).

> Use cryptography. No, not crypto coins! I just mean a bog-standard algorithms like public-private key signing so that it is possible to confirm the source of transactions.

Obvious question then: You've made a card transaction. It is signed with the other party's private key. What does that buy you? How do you attach trust to this key? Whose is it - the payment gateway or the merchant?

> I would much rather have something generated with the assistance of a modern LLM than what we have now, which is security holes big enough to drive a panamax container ship through.

Sure, and if the LLM can rewrite enough of this system to get what you want, there's heaps of room for improvement. But this is orders of magnitude bigger in scope than rewriting your one old bit of COBOL software, it's systematic.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#736

Post by deanishe »

baranul wrote: post #733
NohatCoder wrote: post #702

Earlier quoted context omitted.

There is no such thing as a complete test suite, there will always be some possible bug that it doesn't catch. In particular, if you put an LLM in an automated loop of "this test fails, please fix it", there is a pretty good chance that it will simply special case all of the tests, possibly in some contrived way that makes it not at all obvious when you read the code.

This is what I have observed as well. Been through many debates about the "perfect plan" and "perfect tests" fallacies. Maybe a way of looking at it, to understand the nature of the issue. Have a LLM translate a novel from English to Spanish. Of course it can do that translation at speeds that no human could (score a point for AI). But how good is the Spanish translation? Is the quality better than what humans could…

> how do we know the Spanish was translated perfectly, without a massive detail review

You can't. I think that's a large part of why LLMs have caught on much better with programmers: they have ways of making the computer check its own work.

Checking a document is still a laborious manual task. And completely unfulfilling.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#737

Post by hedgedoops2 »

dzaima wrote: post #725
anhner wrote: post #539

Earlier quoted context omitted.

What an incredibly bad take. "It's not open source because we have the source but not the thought process of the developer" - well then no project on this earth is truly open source by your definition.

Rather depends on definitions; GPL does contain: > The "source code" for a work means the preferred form of the work for making modifications to it. With that definition, there's definitely space for arguing that the AI tooling for modifying the code is necessary for the modification process to be sane therefore "preferable" for any human, if the code is "designed" (or lack of design thereof) around the idea of being…

Excellent point

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#738

Post by baranul »

deanishe wrote: post #736
baranul wrote: post #733

Earlier quoted context omitted.

This is what I have observed as well. Been through many debates about the "perfect plan" and "perfect tests" fallacies. Maybe a way of looking at it, to understand the nature of the issue. Have a LLM translate a novel from English to Spanish. Of course it can do that translation at speeds that no human could (score a point for AI). But how good is the Spanish translation? Is the quality better than what humans could…

> how do we know the Spanish was translated perfectly, without a massive detail review You can't. I think that's a large part of why LLMs have caught on much better with programmers: they have ways of making the computer check its own work. Checking a document is still a laborious manual task. And completely unfulfilling.

And that's the trap. Relying on a "perfectly" crafted test or a LLM to verify what is unknown. Yes, LLMs can be very useful, but perhaps it's best for us to be realistic.

> making the computer check its own work

Kind of like the Spanish teacher telling his students they can grade their own tests, then being surprised that Billy was always giving himself 100%, when he's nowhere near that bright or fluent.

It wouldn't be so bad, if people were more upfront with being unsure or made it clear they were extrapolating from smaller and limited data. But usually, like many of these unusually cocky LLMs, what is too often reported to the public is "perfection" and many inconvenient truths "swept underneath the carpet".

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#739

Post by jiggawatts »

CamouflagedKiwi wrote: post #735
jiggawatts wrote: post #704

Earlier quoted context omitted.

> Because you know that the current one works. What do you even mean by "works", specifically? > it might have bugs, you know what they are. Okay, so it doesn't work, you know it doesn't work, it's just that you accept the specific ways in which it doesn't work. I've lost track of all the myriad stupid ways in which these ancient systems are hugely ineffectual without even being outright faulty. Like airline tickets…

> What do you even mean by "works", specifically? It runs and accepts people's payments, which means you're not on the front page of the newspaper (not in a good way). > ... because they're stored in terrible ancient mainframe databases, the text fields all have tiny maximum lengths. Hence they're all abbreviations. Attacker-controlled abbreviations without any authenticity assurance of any kind! They also have to go…

> It runs and accepts people's payments, which means you're not on the front page of the newspaper (not in a good way).

"It works if nobody attacks it." isn't security.

> They also have to go through payment networks which are very often the limits on those things.

For the same reasons.

> need Visa and Mastercard to upgrade as well.

They won't, and it's not worth asking them to. They're dinosaurs and will simply be replaced by a newer, more agile competitor.

It's already happening! Billions of people in Asia pay with their phones using home-grown payment systems, most of which are generally much more modern and better engineered.

> It is signed with the other party's private key. What does that buy you?

Same as what HTTPS does: attestation of identity by some trusted third-party, to some non-zero level. This could be literally just the existing CA networks and DNS domains as identity, but it could be governments, the banks themselves, etc.

I had some fraudulent transactions on my account labelled "Microsoft Subscription". It wasn't Microsoft. How can I tell?

Not even the bank knew the identity of the third party!

That's insane.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#740

Post by apf6 »

aeonfox wrote: post #728
NohatCoder wrote: post #702

Earlier quoted context omitted.

There is no such thing as a complete test suite, there will always be some possible bug that it doesn't catch. In particular, if you put an LLM in an automated loop of "this test fails, please fix it", there is a pretty good chance that it will simply special case all of the tests, possibly in some contrived way that makes it not at all obvious when you read the code.

This is where fuzzing would be useful. We have an at-least-parity-bug-level oracle with the reference PostgreSQL implementation. Just build a generator of queries (both invalid and valid) and ensure the output matches. The yardstick is how many log10(queries) it can go on average before a discrepancy is found.

It’s 2026, so another basic technique that every team should add to their testing strategy (in addition to proven techniques like fuzzing) is agentic user simulation. Set up an AI agent with access to the product, and prompt it to use the thing in hundreds of realistic use cases, and to report any possible bugs. It will catch a lot of ‘blind spot’ bugs that were previously things that were only caught by humans.
Post reply on HN