I don’t understand these rewrites, honestly, what is the point? Who have had any C++ related issues while working with Postgres?
Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
261–270 of 756 posts
Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
#262Hey author here. Wasn't expecting to see this up. To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture. I'm now working on a new, not yet published version of pgrust that inc…
A thread per connection is a almost always the correct decision for performance, but by choosing a process per connection, postgres is able to let you load whatever sketchy extensions you want. Worst case you crash the process, not the database. It would be nice if you could strike a balance so a segfaul in the extension only crashes a small percentage of connections, not the whole thing.
Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
#263Don’t understand these rewrites. - typically they are behind a single person. That’s usually bad because of spf - typically they are achieved in a very short amount of time, so the author hasn’t acquired any discipline in creating the project. That means it’s unlikely the author is going to stick to the project in the mid and long term - anyone that wants to contribute to the project needs to pay. Needs to pay tokens…
It’s open source. You take ownership of it for your deployment and stop relying on continued free work. You can use llm to pull in updates as they are released. It’s not gpl, so you don’t need to publish your port
The companies I have worked for before all have used open source software like postgres, mysql, go, python, k8s, etc. 99% of the time we relied on free work; never contributed to these projects nor forked them for our own needs. I don’t think this behaviour is the unusual path tbh
Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
#264Don’t understand these rewrites. - typically they are behind a single person. That’s usually bad because of spf - typically they are achieved in a very short amount of time, so the author hasn’t acquired any discipline in creating the project. That means it’s unlikely the author is going to stick to the project in the mid and long term - anyone that wants to contribute to the project needs to pay. Needs to pay tokens…
If you were a token burning company a project like this seems like a solution to this:
https://xkcd.com/2347/
that hits your metrics without the problem that your contributions are not welcome.
Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
#265Earlier quoted context omitted.
I'd %100 prefer an opus 4.8 rewrite over %99 of the time. Unless Fabrice Bellard is rewriting the stuff I need, I'd prefer AI over a human coder.
AI is an average coder. It was trained on all code the code that could be found. Not just code written by genius programmers like Carmack and Bellard. Given that it's average, I'd prefer a human coder above average :)
That is not how it works. IF it was condemned to be average models wouldn't be constantly improving, given that humans aren't getting better.
Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
#266This is impressive - but is a license change, from the PostgresQL license [0] to AGPL [1]. I like the AGPL and think it's the best truly free open source license, but I worry if this is compatible. Ie, if this is rewritten from the original source, should the original apply? (Yes.) There has been a trend to rewrite open source software with a more restrictive license (like coretools in Rust). This looks considerably…
Wow, yeah, really hate this.
Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
#267This is impressive - but is a license change, from the PostgresQL license [0] to AGPL [1]. I like the AGPL and think it's the best truly free open source license, but I worry if this is compatible. Ie, if this is rewritten from the original source, should the original apply? (Yes.) There has been a trend to rewrite open source software with a more restrictive license (like coretools in Rust). This looks considerably…
You seem to have the restrictiveness backwards? The MIT license (uutils coreutils) is less restrictive than the GPL (GNU coreutils), and the AGPL is more restrictive than the PostgreSQL license.
And it doesn't violate the PostgreSQL license to license the rewrite more restrictively. That's part of what makes MIT-style licenses less restrictive than the GPL or AGPL: they allow for more-restrictive relicensing.
Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
#268Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
#269Earlier quoted context omitted.
AI is an average coder. It was trained on all code the code that could be found. Not just code written by genius programmers like Carmack and Bellard. Given that it's average, I'd prefer a human coder above average :)
I dont think Opus 4.8 is an average coder, with my own experience (I have coded 20 + years before even llms existed) it is anything but average. I don't think training data alone determines the success of these models, there are lots of reinforncement learning principles and fine tuning takes place, a crappy code in the dataset doesnt hold those llms scoring high in benchmarks, I dont think an average programmer can…
I think Opus 8.4 is a below average developer, but maybe I have just worked with good developers and have a skewed perspective of what the average is.
Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
#270Hey author here. Wasn't expecting to see this up. To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture. I'm now working on a new, not yet published version of pgrust that inc…
Super impressive! Is it possible for you to share your methodology of using LLMs?
My approach has changed throughout the course of this project. Throughout most of the project, we were working off of a c2rust translation of Postgres to Rust. That gave us a bunch of Rust code that was unsafe but did pass the Postgres test suite and was fast. c2rust had split Postgres into 1000 different crates. We then went through 1 by 1 and rewrote each crate into idiomatic rust.
This naturally lended itself to a suite of skills to describe how to rewrite a crate from unsafe rust to idiomatic rust. The main three skills I had were 1) a skill for identifying the next crates to port 2) a skill for rewriting a crate and 3) a skill for auditing a crate and making sure there weren't any outstanding issues.
My exact approach for managing subagents changed throughout the project. Initially I was doing parallel coding sessions with Conductor. After dynamic workflows came out, I used that as it was really easy to spin up dozens of parallel subagents and manage it from a single orchestrator. Over time I switched from using dynamic workflows to manually spinning up subagents from a central agent. The issue with dynamic workflows is they waterfall. Each step needs to finish before the next one starts. By manually spinning up subagents, I could have claude start porting a new crate as soon as a prior subagent finished.