Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

github.com

Post reply on HN
Print view View original

421–430 of 756 posts

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#421

Post by joshka »

jeltz wrote: post #276
joshka wrote: post #67

Earlier quoted context omitted.

It would be reasonably easy to audit and automate this...

It is not a requirement the PostgreSQL project wants to have. It would be a heavy burden and mostly pointless.

If you examine what you're saying here and slippery slope it a little, examining past effort for misses and correcting them at scale is not worth doing? There are many ways that the second part of your perspective are incorrect (burden / lack of point). I bet that a coding agent could in an automated fasion find at least reasonable additions that you'd say add value and reduce potential for error long term that you'd find valuable. They've probably already done so (I don't know postgres dev at all, so just supposition here. They will 100% do so in the future.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#422

Post by edelbitter »

gingersnap wrote: post #3

I start to see a lot of these re-writes that depend on tests to state that its working. But the things that make software like Postgres and SQLite reliable are not mostly the test, but the real world production scars. That's where the reliability comes from, years and years of running in production.

> That's where the reliability comes from

So, we should make it easier to feed that reliability back upstream.

Probably the most useful thing you can do with these LLM-transpilations for now: If the transpiled version passes all original tests, I can run my application test suite against it and use it to discover test coverage deficiencies in the original!

If it crashes or otherwise observably misbehaves, I know the real project was missing regression tests for something. We could make upstream so much more resilient against accidentally breaking stuff in future updates, if only it becomes safe (offline + no side effects) and easy (if it crashes/locks, it is not from some memory safety bug from 25k transactions earlier) to run these transpiled projects as one row in our everyday integration matrix.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#423

Post by greatony »

malisper wrote: post #147

Hey author here. Wasn't expecting to see this up. To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture. I'm now working on a new, not yet published version of pgrust that inc…

It's a completely new era of software production (I will no longer call it development) LLMs give us unlimited manpower, and the language give us constraints to make more modern and safer softwares. Love to see this rewrite in Rust, and expecting much much more in next few month.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#424

Post by teravor »

malisper wrote: post #147

Hey author here. Wasn't expecting to see this up. To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture. I'm now working on a new, not yet published version of pgrust that inc…

when doing rewrites like these, why isn't the first step to instrument the original code so that you would get very good automated test suites to point the LLM toward?

use both synthetic and real data to sample the internals of the original software to duplicate.

locate all the data transformation junctures, sample and then replicate the tranforms 1:1 in the rewrite.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#425

Post by otterley »

teraflop wrote: post #326
otterley wrote: post #304

Earlier quoted context omitted.

If this software was written by a mechanical process, the license is a nullity. It’s public domain.

Being created by a "mechanical process" from an existing creative work doesn't mean it's not a derivative work. For instance, if I take a copy of $BIG_BUDGET_MOVIE, and resample the video frames from 1080p to 720p through a purely mechanical transformation, that doesn't make the output public domain.

You might be right, if it is a derivative work and not a new one. There’s some evidence that the authors consider it a new one because they’re attempting to change its license.

The BSD license doesn’t explicitly convey the right to create derivative works but it does convey the right to “modify” the software. (They seem similar but “modify” is a narrower verb.) So is this a modification? A derivative work? A new work entirely? If it shares no code with the work from which it was derived, things are more complex than it may seem (and it no longer fits the “compressed video” analogy very well).

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#426

Post by rirze »

maxloh wrote: post #121
juliangmp wrote: post #2

I feel like we need to heavily differentiate between a rewrite and an AI rewrite.

For instance, the TypeScript rewrite in Go was done mostly by humans and took a year before it was released. That is how you rewrite software that people can trust.

> mostly by humans

`mostly` is doing a lot lifting here. The Go rewrite uses plenty of copilot. The reason you trust it is because you trust the people doing the rewrite.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#427

Post by dboreham »

maxloh wrote: post #121
juliangmp wrote: post #2

I feel like we need to heavily differentiate between a rewrite and an AI rewrite.

For instance, the TypeScript rewrite in Go was done mostly by humans and took a year before it was released. That is how you rewrite software that people can trust.

Many projects that were done by humans and took a year can certainly not be trusted.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#428

Post by kardianos »

jeltz wrote: post #327
kardianos wrote: post #309

Earlier quoted context omitted.

PG Wire proto 3 is my largest source of frustrations. I'm playing with a POC for a better wire protocol here: https://github.com/solidcoredata/pgwire4

404 Not Found

Now public.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#429

Post by ori_b »

fpgaminer wrote: post #292
film42 wrote: post #262

Earlier quoted context omitted.

A thread per connection is a almost always the correct decision for performance, but by choosing a process per connection, postgres is able to let you load whatever sketchy extensions you want. Worst case you crash the process, not the database. It would be nice if you could strike a balance so a segfaul in the extension only crashes a small percentage of connections, not the whole thing.

If it's a choice between performance and being able to "safely" run sketchy extensions, I'd rather have performance.

But it isn't.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#430

Post by airstrike »

rvz wrote: post #406
kirubakaran wrote: post #254

Earlier quoted context omitted.

Everything has to start somewhere

How is that going for Claude's C compiler [0], or Cursor's web browser? [1] [0] https://github.com/anthropics/claudes-c-compiler [1] https://github.com/wilsonzlin/fastrender

Terribly, but just because any two people suck at basketball doesn't mean a third person must suck too.
Post reply on HN