Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

github.com

Post reply on HN
Print view View original

601–610 of 756 posts

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#601

Post by LtWorf »

alemanek wrote: post #340
gingersnap wrote: post #3

I start to see a lot of these re-writes that depend on tests to state that its working. But the things that make software like Postgres and SQLite reliable are not mostly the test, but the real world production scars. That's where the reliability comes from, years and years of running in production.

Not weighing in on this specific rewrite but tests are how you specify that your software works correctly. If a behavior isn’t covered by an automated test in some form you can’t assert that any given change doesn’t break it. I think it is completely reasonable to use a preexisting unmodified test suite to state that something is working. The larger the project the more true this becomes. Real world production scars…

But they aren't all open, so your llm rewrite/copyright eraser won't be taking advantage of them all.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#602

Post by groundzeros2015 »

malisper wrote: post #165
eu-tech-tak wrote: post #72

How is the performance compared to regular PostgreSQL? I know it says it is not performance optimized yet, but if this succeeds, will it only bring more "memory safety" or is there a serious performance gain as well?

The version in the GitHub repo is ~8x slower than Postgres. I have a new unpublished version that is 50% faster than Postgres on transactional workloads and ~300x faster on analytical workloads.

I don’t think that’s true.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#603

Post by groundzeros2015 »

ZiiS wrote: post #26

What would be interesting is if they found a memory unsafe bug. Postgres is a perfect case study of 30 years of C with a bit of CPP; if rewriting in a safer language didn't find anything...

C programmer have learned how to deal with memory problems and have whole suites of tools for finding them.

Is it cheaper to find them at compile time rather than runtime? Yes.

But it’s not an unsolved problem. Memory bugs are a known unknown.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#604

Post by groundzeros2015 »

evil-olive wrote: post #107

> The goal is to make Postgres easier to change from the inside uh-huh, sure. you want to show off "look what the LLM can do / look what I burned a bunch of tokens on"? you want to brag about how your LLM-generated slop is somehow more maintainable than the original because blah blah blah Rust? here [0] is the version history of Postgres. pick a version from the past. let's say 14.x because it's the most current that…

This is actually a great point I have not seen articulated. A motivation for clear architecture and good design is being able to evolve the system over time. Does LLM code have that capacity? Are LLMs themselves able to increment iteratively?

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#605

Post by literalAardvark »

sgt wrote: post #599
happyweasel wrote: post #506

Porting perfectly working C code that has been in production for decades (and has a great track record wrt to security fixes) to rust is more or less an obfuscation challenge.

An academic exercise for sure. The Postgres team won't use this and take it forward, hence it will go stale and rot within months.

Or someone who likes it could run codex-get -y upgrade on it once every two weeks and it'll be fine for as long as you can afford the tokens.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#606

Post by bluGill »

teravor wrote: post #424
malisper wrote: post #147

Hey author here. Wasn't expecting to see this up. To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture. I'm now working on a new, not yet published version of pgrust that inc…

when doing rewrites like these, why isn't the first step to instrument the original code so that you would get very good automated test suites to point the LLM toward? use both synthetic and real data to sample the internals of the original software to duplicate. locate all the data transformation junctures, sample and then replicate the tranforms 1:1 in the rewrite.

That forces you not only in not the intentional good decisions of the past but also copies too many bad ones.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#607

Post by bluGill »

wongarsu wrote: post #564
otterley wrote: post #433

Earlier quoted context omitted.

IAAL (not legal advice) and I’m not sure the issue is settled. The BSD license only explicitly permits the author “to use, copy, modify, and distribute this software and its documentation for any purpose, without fee, and without a written agreement.” By default, the owner of a protected work retains all rights not conveyed to someone else. Changing the license isn’t one of the enumerated activities, and so I think t…

You are not really changing the license of the original work though. You are not impacting what the author or anyone else can do. You are distributing a copy under different, more restrictive terms. A bit like me buying a comic book, then offering to sell it to you under the condition that you never let my brother read it and that you make any future owner agree to the same terms. That's perfectly legal, and there is…

Your redistribution license only applies to the part you own copyright on. Everything else is still under the original license. You can add AGPL but that doesn't take away the BSD license from the parts you didn't create.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#608

Post by bluGill »

otterley wrote: post #425
teraflop wrote: post #326

Earlier quoted context omitted.

Being created by a "mechanical process" from an existing creative work doesn't mean it's not a derivative work. For instance, if I take a copy of $BIG_BUDGET_MOVIE, and resample the video frames from 1080p to 720p through a purely mechanical transformation, that doesn't make the output public domain.

You might be right, if it is a derivative work and not a new one. There’s some evidence that the authors consider it a new one because they’re attempting to change its license. The BSD license doesn’t explicitly convey the right to create derivative works but it does convey the right to “modify” the software. (They seem similar but “modify” is a narrower verb.) So is this a modification? A derivative work? A new work…

The law and courts are not really settled here from what I can tell. I expect some day hundreds of millions $$$ will be spent on this

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#609

Post by KolmogorovComp »

malisper wrote: post #270
solomatov wrote: post #247

Earlier quoted context omitted.

Super impressive! Is it possible for you to share your methodology of using LLMs?

My approach has changed throughout the course of this project. Throughout most of the project, we were working off of a c2rust translation of Postgres to Rust. That gave us a bunch of Rust code that was unsafe but did pass the Postgres test suite and was fast. c2rust had split Postgres into 1000 different crates. We then went through 1 by 1 and rewrote each crate into idiomatic rust. This naturally lended itself to a…

Thanks for the insights, are the skills available anywhere?

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#610

Post by lucideer »

davedx wrote: post #584
sdevonoes wrote: post #251

Don’t understand these rewrites. - typically they are behind a single person. That’s usually bad because of spf - typically they are achieved in a very short amount of time, so the author hasn’t acquired any discipline in creating the project. That means it’s unlikely the author is going to stick to the project in the mid and long term - anyone that wants to contribute to the project needs to pay. Needs to pay tokens…

This is very critical of an open source project that the maintainer didn't even post here? "Status: pgrust is not production-ready yet. It is not performance optimized yet." The maintainer is not suggesting you use this for anything yourself. So why do you care about spf or (lol) his "discipline in creating the project"?

The gp feels like less of a targeted individual criticism & more of a general musing about a trend.

Nobody is saying this author hasn't demonstrated discipline & won't maintain this project, but the statistical averages across most projects fitting this trend make it likely enough to question their worth in aggregate.

Post reply on HN