Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

github.com

Post reply on HN
Print view View original

671–680 of 756 posts

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#671

Post by wkoszek »

pknerd wrote: post #44

I am not trolling, but I have a simple question: Why? Why do I use this instead of the official build? What is the business case?

There is big appetite for PostgreSQL in business cases. But there's also a lot of problems in PG and people want to solve them. But there's ~10 people in the core of PostgreSQL who contribute to it and know how to change the core. If you have a business use case that would require changing the core, doing it in safer, less error prone technology would be way better. That's why you have other products that had to be created that talk PG protocol, but aren't using official build.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#672

Post by cyber1 »

atombender wrote: post #648
cyber1 wrote: post #182

2664 "unsafe {", 1835 "unsafe fn". This is completely unsafe. It doesn't look like a rewrite that understands what's actually going on or how the architecture should be redesigned to take advantage of Rust strengths. Instead, it looks like an AI generated transpilation with extensive use of raw pointers.

Counterpoint: All of the current Postgres codebase is already wrapped in an invisible unsafe{}. The difference with a Rust codebase like this is that all of the unsafe code has been neatly isolated and clearly marked. The outside code is safe — at least according to the definition of what Rust considers safe, which is a high bar indeed and objectively superior to the unsafe mess that is C — and the unsafe code is nat…

Valid point, but unsafe in Rust is more dangerous than unsafe in C, bcs of aliasing. For example PG is compiled with -fno-strict-aliasing.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#673

Post by cestith »

otterley wrote: post #304
vintagedave wrote: post #195

This is impressive - but is a license change, from the PostgresQL license [0] to AGPL [1]. I like the AGPL and think it's the best truly free open source license, but I worry if this is compatible. Ie, if this is rewritten from the original source, should the original apply? (Yes.) There has been a trend to rewrite open source software with a more restrictive license (like coretools in Rust). This looks considerably…

If this software was written by a mechanical process, the license is a nullity. It’s public domain.

It wasn’t written by a mechanical process, though. It was ported (translated) from an existing creative work into a new language.

Surely you wouldn’t say a Spanish translation of a Harry Potter work is in the public domain while the original work is under copyright?

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#674

Post by menaerus »

mort96 wrote: post #512
malisper wrote: post #318

Earlier quoted context omitted.

> Threads does not offer any major performance advantage This is very not true. When it comes to parallel queries, a process model adds a ton of overhead. You can't pass pointers between processes because the address space is different. This adds a ton of overhead in a bunch of different places. For example when doing a parallel hash join, Postgres will have each worker build a local hash table. Then it will take all…

Passing pointers is not significantly faster than passing offsets into a shared memory pool.

Sorry, what? Passing a pointer is a matter of wrapping the value into the CPU register. OTOH passing an offset into a shared memory is a write to main memory so several magnitudes slower.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#675

Post by athanagor2 »

jiggawatts wrote: post #153
CamouflagedKiwi wrote: post #133

Earlier quoted context omitted.

I have some familiarity with the bank situation, and while a lot of them are on some very old systems (maybe COBOL, maybe something else, either way they want off it) the cost of actually re-writing the code is far from the most significant issue. Consider: You have a big mainframe running your tier 1 bank. Assume that you can see all the code on it, and you can feed all that to an LLM if you like. Getting it to spit…

> while you could ask your LLM to do that you aren't going to run your bank on the result. Why not? I feel like we're entering a new era of prejudice against not a category of humans, but against non-human intelligences. The design patterns for distributed and fault-tolerant systems are well-known and established in the industry. Both humans and AIs are familiar with them! So if you sketch a design for the AI to foll…

How did you conduct this rewrite? Did you hand the AI some specs, some tests, the existing code?

I feel like AI has dramatically changed how complete rewrites can be considered, especially for long-lived, legacy projects.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#676

Post by wkoszek »

Congrats for launching this project. I think it's awesome. I did two similar projects, just to learn how to work with LLMs and it felt good--reminded me times debugging code with GDB and going through stuff in semi-manual mode.

I'm building data + AI platform. It got complex, and I'm using AI-assisted coding to move fast. One thing that helps me was property based testing. I have a traffic generator that simulates 10 users working on my platform. I run it 24/7 and if it shows that the software survives the test (I called it "fate" after ffmpeg's CI), it's good enough to roll out. If you wrote something like that, core PostgreSQL folks would like it too, unless there's something equivalent like this. It'd be: create random tables, fill with random data, then issue randomly constructed query.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#677

Post by otterley »

cestith wrote: post #673
otterley wrote: post #304

Earlier quoted context omitted.

If this software was written by a mechanical process, the license is a nullity. It’s public domain.

It wasn’t written by a mechanical process, though. It was ported (translated) from an existing creative work into a new language. Surely you wouldn’t say a Spanish translation of a Harry Potter work is in the public domain while the original work is under copyright?

I would agree with you but for the author’s attempt to publish it under a new license. I think they can either claim it’s a new work (in which case it’s public domain) or claim it’s a derivative work (in which case I don’t think they can change the license).

I imagine a court would call it a derivative work if tested.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#678

Post by bwfan123 »

ottavio wrote: post #33

Why should a developer use this for anything beyond a pet project? Just because it is written in Rust? All these "rewritten in rust" projects only reinforce the idea that a significant part of the rust community consists of software talibans and not of engineers who must deliver something that works and is reliable over time.

> All these "rewritten in rust" projects only reinforce the idea that a significant part of the rust community consists of software talibans and not of engineers who must deliver something that works and is reliable over time.

Nailed it ! There are some folks who behave holier-than-thou just because they happen to use some language. Language missionaries if you will, and they are insufferable.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#679

Post by HumblyTossed »

wtetzner wrote: post #656
HumblyTossed wrote: post #649

Earlier quoted context omitted.

> One of the things I'd typically do is peek at the commit history. Seeing what people worked on and how they did it tends to say a lot about a project I could not care less about any of this. Truth is code, as it is now. I don't care when (and certainly not by who) a bug got introduced, it's here, shut up and fix it.

So your solution is to just read through all 1M+ lines of code?

And yours is to read through all 37000 commit messages??

don't be obtuse. Nobody needs to do either.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#680

Post by cmrdporcupine »

wkoszek wrote: post #665
cmrdporcupine wrote: post #629

Earlier quoted context omitted.

Came here to the same thing. I did something similar (but bolder, it doesn't slavishly copy Postgres and is based on current DB research papers and the like, and other bits I've been exposed to over the years). It has a full TPC-C-esque benchmark suite, replication, embedded v8/JavaScript relations/stored procedures, a giant suite of regression tests, it kicks the crap out of a lot of existing OLTP DB stuff out there…

Author of this project is not that different than you. The only difference is that he is willing to take the risk. If you have an invention and sit on it, nothing will happen.

That's not it works anymore. Inventing is cheap.
Post reply on HN