Live data from Hacker News

Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

github.com

721–730 of 756 posts

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#721

Earlier quoted context omitted.

Being really pedantic here, shared memory is considered IPC, but not the kind you're thinking of. Shared address space, no overhead.

"Shared address space, no overhead" But concurrent access, so synchronization is required (lock or whatever), so overhead :)

Yeah that's true, and I'd say threads need synchronization for concurrent access too, but supposedly the options for doing that are faster than what you need to use across processes.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#722
post #682

Earlier quoted context omitted.

Let me just point out: The project is not cool. This is not a new idea, and there is nothing special. Students won't use it as an example of porting code. I am not aware "porting code" is part of standard software engineering or computer science curriculum. That's not the kind of thing being taught in schools. Companies won't switch to it unless their CTOs are either insane or incompetent.

If someone did this before with PG, provide links. Practical pragmatic example where I think this project is cool: process vs thread. There were members of the PG core team that wanted to explore it, and members who said it'd tank a project. pgrust is an amazing experimentation ground for it. I published similar project here: www.emuko.dev - emulator for RISC-V. This one turned out to be 3x as slow as QEMU for exampl…

It's a matter of whether this project will provide bug fixes, will continue to exist next month and whether the company will end up scrambling to replace it. No CTO cares about (very) questionable "improvements". They care about if it works and will continue to work.

Sorry, as someone who has been involved in many of these decisions, I don't think you understand how any of this actually works in the real world.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#723
post #722

Earlier quoted context omitted.

If someone did this before with PG, provide links. Practical pragmatic example where I think this project is cool: process vs thread. There were members of the PG core team that wanted to explore it, and members who said it'd tank a project. pgrust is an amazing experimentation ground for it. I published similar project here: www.emuko.dev - emulator for RISC-V. This one turned out to be 3x as slow as QEMU for exampl…

It's a matter of whether this project will provide bug fixes, will continue to exist next month and whether the company will end up scrambling to replace it. No CTO cares about (very) questionable "improvements". They care about if it works and will continue to work. Sorry, as someone who has been involved in many of these decisions, I don't think you understand how any of this actually works in the real world.

I mean.. they made it 2 weeks ago. Clearly the time isn't now. Why be so negative immediately.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#724

Earlier quoted context omitted.

The alternative is that you are not easily replaceable, which means that moi2389 is the one who doesn't get to do the work. Which is good for moi2388, perhaps, but what about moi2389? Either way someone is going to be left out in the cold.

moi2389 should research the market and create a new value, not replace someone else by providing the same value.

Same as what moi2388 would do in the opposite scenario. Six of one, half dozen of the other.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#725
post #539

Earlier quoted context omitted.

In general (I’m not saying this is the case with this project) if you don’t have their prompt history and you can’t re-run the LLM “compilation” yourself, is it open source? It feels a bit more like those “source available” projects where you can read the code but don’t have access to the build system. On the other hand, aside from the commit messages, one didn’t ever have access to the underlying thought process of…

What an incredibly bad take. "It's not open source because we have the source but not the thought process of the developer" - well then no project on this earth is truly open source by your definition.

Rather depends on definitions; GPL does contain:

> The "source code" for a work means the preferred form of the work for making modifications to it.

With that definition, there's definitely space for arguing that the AI tooling for modifying the code is necessary for the modification process to be sane therefore "preferable" for any human, if the code is "designed" (or lack of design thereof) around the idea of being AI-maintained.

Otherwise, it's not source-code, it's not meaningfully-modifiable, it's basically equivalent to just decompiling a binary. (similarly-bad quality may of course be human-produced too, though then at least you have direct proof of it being the preferred form for at least one person - the author)

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#726
This the type of AI generated tool I'm interested in seeing more of. I'm a fan of how the quality is verified using clearly defined quantitative measurements for performance and correctness.

I'm still skeptical about LLMs and don't use them, although I can be convinced by more demonstrated examples of success.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#727

Hey author here. Wasn't expecting to see this up. To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture. I'm now working on a new, not yet published version of pgrust that inc…

I don't want to knock you down as most have already did. In-fact it's a useful exercise going forward in exploring how to work with AI. It's here, we're all going to use it one way or the other. Zero issues with that, in-fact kudos to going through the pain of it all. Now, having gone through several such endeavors originally myself, albeit with internal tools and systems (as an exercise), I've noticed that while all…

I don't code with LLMs, and think you might be right.

However, Postgres is a tool with clearly defined functionality and doesn't have ambiguous requirements that are seen in user facing software. As a result, it is entirely plausible that the author created a working Postgres replacement for certain use cases.

I personally want to see more evidence about the quality of the tool after it is in a finished state.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#728

Earlier quoted context omitted.

I'm curious. Do you attribute this to weak and/or incomplete tests? How granular should tests be to have complete coverage so that an AI won't create a converted codebase that "passes tests" but is still functionally inaccurate?

There is no such thing as a complete test suite, there will always be some possible bug that it doesn't catch. In particular, if you put an LLM in an automated loop of "this test fails, please fix it", there is a pretty good chance that it will simply special case all of the tests, possibly in some contrived way that makes it not at all obvious when you read the code.

This is where fuzzing would be useful. We have an at-least-parity-bug-level oracle with the reference PostgreSQL implementation. Just build a generator of queries (both invalid and valid) and ensure the output matches. The yardstick is how many log10(queries) it can go on average before a discrepancy is found.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#729

Earlier quoted context omitted.

I'm curious. Do you attribute this to weak and/or incomplete tests? How granular should tests be to have complete coverage so that an AI won't create a converted codebase that "passes tests" but is still functionally inaccurate?

That's the million dollar question. Do you/we/us have tests that cover everything which covers QA as well? If such a mythical beast exists, maybe from remnants of ye olde TDD past and hasn't been modified as such.. then maybe this would be possible to do as such.

I've always played the game that tests are only as useful as you have resources to attribute to them. Mostly all modern development is a compromise between new features and stuff that supports those new features (tests included, but also reviews, code maintenance, docs, etc.).

If LLMs can be utilized to quickly make deep testing possible, I think that's probably a net-positive.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#730

Earlier quoted context omitted.

I don't want to knock you down as most have already did. In-fact it's a useful exercise going forward in exploring how to work with AI. It's here, we're all going to use it one way or the other. Zero issues with that, in-fact kudos to going through the pain of it all. Now, having gone through several such endeavors originally myself, albeit with internal tools and systems (as an exercise), I've noticed that while all…

I'm curious. Do you attribute this to weak and/or incomplete tests? How granular should tests be to have complete coverage so that an AI won't create a converted codebase that "passes tests" but is still functionally inaccurate?

Unit tests don't test for branch coverage.

That's the culprit, because LLMs tend to forget and remove a lot of branch logic in these kinds of tasks. If unit tests don't cover these specific if/elseif/else cases, then they'll just disappear.

They'll also disappear if the LLM is allowed to modify the unit tests, because they sure like to cheat their way around into greenlit test suites. The agentic environment must disallow write access to the unit test files for the agent that writes the code.

If you implement that in your tools, you'll see quickly how the models will try to rewrite the unit tests at all cost, no matter what kind of prompting you've done. Tool policies are the only boundary to successfully guarantee this.

Source: Am building my own agentic environment because of that behavior

Post reply on HN