The code is so weird, people will have a hard time reading it when they need to check for correctness.
There are much better ways to write it in Rust: https://github.com/malisper/pgrust/blob/14ffab7d31a31e5ab667...
Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
481–490 of 756 posts
Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
#482Earlier quoted context omitted.
A thread per connection is a almost always the correct decision for performance, but by choosing a process per connection, postgres is able to let you load whatever sketchy extensions you want. Worst case you crash the process, not the database. It would be nice if you could strike a balance so a segfaul in the extension only crashes a small percentage of connections, not the whole thing.
If it's a choice between performance and being able to "safely" run sketchy extensions, I'd rather have performance.
While PG's behavior doesn't guarantee a lack of data corruption, "an extension crashed, all bets are off, tear everything down" is going to give you a much better fighting chance against data corruption vs the alternative.
Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
#483Don’t understand these rewrites. - typically they are behind a single person. That’s usually bad because of spf - typically they are achieved in a very short amount of time, so the author hasn’t acquired any discipline in creating the project. That means it’s unlikely the author is going to stick to the project in the mid and long term - anyone that wants to contribute to the project needs to pay. Needs to pay tokens…
It's a very interesting phenomenon in recent years and most of the discussions about "why" are immediately blocked by the "memory safety" argument, as if it's a silver bullet for all things that are considered "bad" in software implementations.
No matter how good the language is, and I consider Rust a very good language, it's practically impossible to replace years of experience and tested code, most of it contributed by a ton of brilliant programmers, no matter how you look at it.
And if we take this as the truth, then logically there's still an unanswered question - why? Lets take an undeniable fact - rewriting an existing project give you full control over the new implementation. You can do whatever you want with it, you can't be sued. The only thing now you have to hope for is for your implementation to gather a good enough user base and from then on you can practically hijack the original project.
And this is me speculating - rewriting stuff in Rust isn't about the greater good for that magical "memory safety" argument, but at the end it's an attempt to hijack popular software projects.
Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
#484I suspect the future of open source will be to never publish your tests. Or someone will just pump them into an LLM like this.
SQLite already does this but hasn’t stopped the rust /go clones from popping up . These are toy projects with no serious interest in maintaining the port long term. and even with things like bun where the port is merged it remains to be seen on maintainability over time.
I'm glad there's a go clone for SQLite, it' easier to integrate into a Go project, especially when you have to support multiple platforms.
Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
#485Hey author here. Wasn't expecting to see this up. To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture. I'm now working on a new, not yet published version of pgrust that inc…
Do you have anything in the regression test suite like jepsen etc?
Nothing major yet. Once I wrap up the performance work I'm doing I'll start looking at the best way to go about testing. I suspect there's a lot of novel things you can do with agents.
Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
#486Hey author here. Wasn't expecting to see this up. To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture. I'm now working on a new, not yet published version of pgrust that inc…
Just a couple of ideas if you run out of backlog:)
- proper versionnumber (64bit)
- native json streaming. It would be awesome to get to the point where i could somehow redirect the sql output to the browser directly, but piping will do for now. The idea is to be able to stream rows to client without caching and building json along the way.
Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
#487finally, a database that will refuse to compile your schema migration because you used an unsafe join. progress.
Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
#488That's actually interesting - gives C developers motivation to improve postgresql. after all people could say "look, Rust makes this easier".
Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
#489Earlier quoted context omitted.
Some of this post reminds me of a story I heard long ago from someone who had worked at a HW/SW company. They’d transferred an engineer from the ASIC design team to the OS kernel team, though he’d never been on a software team before. After a while the manager called him in for the following conversation: Manager: You’re doing amazing work — zero bugs in production! I’d like you to mentor the other SWEs on how to get…
Funny story but in my experience hardware engineers produce some of the worst software of the industry. Of course there must be some hardware engineers out there who do hood software but generally what they build are disasters.
Honestly, i do not blame them for that though. The whole eco-system of in C- procedurally bitbang on some registers and read on others, until some circuit that might be there is coerced into doing work - often with faulty prototypes you have to rewire yourself, whos documentation is - non-existant-complete while having deadlines within deadlines. And the project-culture is just "aggregate" a layer, wrap the problem like a pearl in shellacks as the main abstraction.
Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests
#490Hey author here. Wasn't expecting to see this up. To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture. I'm now working on a new, not yet published version of pgrust that inc…
i highly doubt you can make it faster than clickhouse, but happy to see it.