Super impressive! Is it possible for you to share your methodology of using LLMs?
My approach has changed throughout the course of this project. Throughout most of the project, we were working off of a c2rust translation of Postgres to Rust. That gave us a bunch of Rust code that was unsafe but did pass the Postgres test suite and was fast. c2rust had split Postgres into 1000 different crates. We then went through 1 by 1 and rewrote each crate into idiomatic rust. This naturally lended itself to a…
I think I would be horrified looking at your Claude API bill.
> I start to see a lot of these re-writes that depend on tests to state that its working. There's another way to validate the rewrite though. Just run both pgrust and postgres and compare the output. Know of an edge case? Run it too. Doesn't know? Use a fuzzer or some automated tool to find interesting inputs. Found an inconsistency? The input/output pair becomes a test case now Not sure if there's tooling for that t…
> Just run both pgrust and postgres and compare the output. The space of inputs and outputs is infinite. You can't prove programs are the same by "just" testing a bunch inputs.
Indeed. This approach is an improvement / augmentation over tests, not a panacea.
Neither postgres nor pgrust have their behavior specified using formal methods. (pgrust could write some contracts using something like kani or creusot, but having upstream postgres also write contracts is a tougher sell). If they had, one could write a giant proof that said the two software essentially do the same thing (at least in a subset of environments and some simplifying assumptions)
> I start to see a lot of these re-writes that depend on tests to state that its working. There's another way to validate the rewrite though. Just run both pgrust and postgres and compare the output. Know of an edge case? Run it too. Doesn't know? Use a fuzzer or some automated tool to find interesting inputs. Found an inconsistency? The input/output pair becomes a test case now Not sure if there's tooling for that t…
(I'm working with malisper on this) we built this too and are using it for a new version we're working on right now
How would one go about reviewing a piece of code like this? One of the things I'd typically do is peek at the commit history. Seeing what people worked on and how they did it tends to say a lot about a project. But with LLMs generating 7101 commits in less than a month that isn't feasible. Even looking at a single day is way too much [1]. It probably also doesn't make sense since the commits content won't tell you mu…
> ps. How do you easily get to the first commit in a repo on GitHub? Browsing commit history feels rather tedious I usually check the history of a file not easily changed like .gitignore. The first commit seems to be this one https://github.com/malisper/pgrust/commit/22113dc36b02973060...
How is that going for Claude's C compiler [0], or Cursor's web browser? [1] [0] https://github.com/anthropics/claudes-c-compiler [1] https://github.com/wilsonzlin/fastrender
Terribly, but just because any two people suck at basketball doesn't mean a third person must suck too.
malisper is the LeBron James of porting Postgres to Rust!
What about extensions that are not sketchy? Lots of good ones out there.
Presumably they'd be fine running in a threaded context.
An extension written for a single threaded host system might not work in a multi-threaded context. For example if has global or shared state that isn't protected with locks or similar (which is unfortunately fairly common in c code)
How would one go about reviewing a piece of code like this? One of the things I'd typically do is peek at the commit history. Seeing what people worked on and how they did it tends to say a lot about a project. But with LLMs generating 7101 commits in less than a month that isn't feasible. Even looking at a single day is way too much [1]. It probably also doesn't make sense since the commits content won't tell you mu…
You don’t. You trust that passing the regression tests means you are totally compatible with the original version.
A thread per connection is a almost always the correct decision for performance, but by choosing a process per connection, postgres is able to let you load whatever sketchy extensions you want. Worst case you crash the process, not the database. It would be nice if you could strike a balance so a segfaul in the extension only crashes a small percentage of connections, not the whole thing.
Some see a 30 year old system and think "outdated", I see a 30 year old system and think "time tested." Clearly a process per connection is more stable and that's what I'm using. It's unclear what problem such optimizations are solving anyway, with the old way you could only support a million concurrent users with a single server? Are we missing out on supporting ten million concurrent users with 2 servers instead of…
A million users? Hell, I'd bet 99.999% of live postgres databases in existence serve less than 5 users on average. Even among products that actually make a profit, I bet 99.9% of them serve less than 100 customers a day. We hooligans on hacker news manage the 0.1% of databases, and in my newfound consulting life, I'm hoping to never support one of those again.
How is that going for Claude's C compiler [0], or Cursor's web browser? [1] [0] https://github.com/anthropics/claudes-c-compiler [1] https://github.com/wilsonzlin/fastrender