Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

github.com

Post reply on HN
Print view View original

301–310 of 756 posts

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#301

Post by LtWorf »

fragmede wrote: post #93
pknerd wrote: post #44

I am not trolling, but I have a simple question: Why? Why do I use this instead of the official build? What is the business case?

Because Rust is what's cool these days. Don't you wanna be cool? Also Rust has memory safety things that C++ doesn't have, so there's a class of bugs that can't happen in the Rust version. That doesn't mean the Rust version is 100% bug free, but just that it's not vulnerable to that class of bugs. So it's a good thing for security reasons if you're running a database server somewhere that attackers could get at it. T…

Rust doesn't have ACID and I'm sure this doesn't either.

I'd like to know if the "authors" know what I'm talking about.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#302

Post by quadrature »

malisper wrote: post #278
quadrature wrote: post #272

Earlier quoted context omitted.

Are you fixing the heap and table management ?. Postgres does not use an undo log and manages all table updates directly in table storage which slows MVCC. also have you told Ben Dicken ? https://x.com/BenjDicken/status/2074326407795417435

That's something I eventually want to fix. The challenge is the storage format is so integral to Postgres that it's going to be a huge PITA to come up with a novel design. Right now OrioleDB is in beta. Once that becomes production ready, I'll evaluate incorporating it into pgrust. For Ben Dicken, he has seen the project: https://x.com/BenjDicken/status/2074512043462603236 . We're still working on all the novel featu…

best of luck to you, it will be great to see a rearchitected postgres.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#303

Post by jeltz »

fpgaminer wrote: post #292
film42 wrote: post #262

Earlier quoted context omitted.

A thread per connection is a almost always the correct decision for performance, but by choosing a process per connection, postgres is able to let you load whatever sketchy extensions you want. Worst case you crash the process, not the database. It would be nice if you could strike a balance so a segfaul in the extension only crashes a small percentage of connections, not the whole thing.

If it's a choice between performance and being able to "safely" run sketchy extensions, I'd rather have performance.

Threads does not offer any major performance advantage, performance of processes vs threads is virtually the same. The reason the PostgreSQL project is moving towards threads is to make development easier.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#304

Post by otterley »

vintagedave wrote: post #195

This is impressive - but is a license change, from the PostgresQL license [0] to AGPL [1]. I like the AGPL and think it's the best truly free open source license, but I worry if this is compatible. Ie, if this is rewritten from the original source, should the original apply? (Yes.) There has been a trend to rewrite open source software with a more restrictive license (like coretools in Rust). This looks considerably…

If this software was written by a mechanical process, the license is a nullity. It’s public domain.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#305

Post by malisper »

OsrsNeedsf2P wrote: post #291
malisper wrote: post #147

Hey author here. Wasn't expecting to see this up. To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture. I'm now working on a new, not yet published version of pgrust that inc…

What's your actual background and expertise with Postgres and databases more broadly? Basically, do you actually know what you're doing, or is there likely a massive footgun you don't know or haven't shared with us?

I spent a couple years managing a Postgres cluster with a petabyte of data. I wrote a couple blog posts from my work then[0][1]. I also wrote dozens of posts on the Postgres internals[2]. I've also given talks on how to generate fractals with SQL[3] and how to write a lisp interpreter in SQL[4].

[0] https://www.heap.io/blog/testing-database-changes-right-way

[1] https://www.heap.io/blog/analyzing-performance-millions-sql-...

[2] https://malisper.me/table-of-contents/

[3] https://www.youtube.com/watch?v=xKoYIvMFnoQ

[4] https://www.youtube.com/watch?v=MPSMH8w7nfw

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#306

Post by krashidov »

malisper wrote: post #270
solomatov wrote: post #247

Earlier quoted context omitted.

Super impressive! Is it possible for you to share your methodology of using LLMs?

My approach has changed throughout the course of this project. Throughout most of the project, we were working off of a c2rust translation of Postgres to Rust. That gave us a bunch of Rust code that was unsafe but did pass the Postgres test suite and was fast. c2rust had split Postgres into 1000 different crates. We then went through 1 by 1 and rewrote each crate into idiomatic rust. This naturally lended itself to a…

Did you hand write the skills or did you have an agent audit your work and infer patterns?

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#307

Post by LtWorf »

shawabawa3 wrote: post #155
ottavio wrote: post #33

Why should a developer use this for anything beyond a pet project? Just because it is written in Rust? All these "rewritten in rust" projects only reinforce the idea that a significant part of the rust community consists of software talibans and not of engineers who must deliver something that works and is reliable over time.

> Why should a developer use this for anything beyond a pet project? If it _is_ 50% faster, then that's the reason Obviously like any new database it's very risky to use so probably only used for niche use cases at first, but if it turns out to be just as reliable as postgres and faster then why not?

It probably is 50% faster, probably by skipping flushing data to disk.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#308

Post by codesnik »

jeltz wrote: post #303
fpgaminer wrote: post #292

Earlier quoted context omitted.

If it's a choice between performance and being able to "safely" run sketchy extensions, I'd rather have performance.

Threads does not offer any major performance advantage, performance of processes vs threads is virtually the same. The reason the PostgreSQL project is moving towards threads is to make development easier.

unless you're spawning them for new connections.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#309

Post by kardianos »

malisper wrote: post #147

Hey author here. Wasn't expecting to see this up. To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture. I'm now working on a new, not yet published version of pgrust that inc…

PG Wire proto 3 is my largest source of frustrations.

I'm playing with a POC for a better wire protocol here: https://github.com/solidcoredata/pgwire4

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#310

Post by signatoremo »

pknerd wrote: post #44

I am not trolling, but I have a simple question: Why? Why do I use this instead of the official build? What is the business case?

Why not? The author have their own reasons to do it. Did they ask you to use it instead of the official build? It's a github repo.

Why does there need to be a business case? They aren't selling it.

Post reply on HN