Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

github.com

Post reply on HN
Print view View original

511–520 of 756 posts

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#511

Post by mulmen »

sevg wrote: post #509
mulmen wrote: post #434

Earlier quoted context omitted.

I performed both searches on Kagi and didn't see anything about IBM. I do see results for "DataBlade" and "Data blade database" but I didn't try those specific variations after my first two attempts returned nothing of interest. That's too much effort to decipher a HN post.

> That's too much effort to decipher a HN post. It would have been less effort than it took you to write this comment. Perhaps next time that you can’t be bothered, just ignore the comment and move on to another thread that meets your required spoonfeeding levels.

Yeah you're absolutely correct. Too late to edit/delete.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#512

Post by mort96 »

malisper wrote: post #318
jeltz wrote: post #303

Earlier quoted context omitted.

Threads does not offer any major performance advantage, performance of processes vs threads is virtually the same. The reason the PostgreSQL project is moving towards threads is to make development easier.

> Threads does not offer any major performance advantage This is very not true. When it comes to parallel queries, a process model adds a ton of overhead. You can't pass pointers between processes because the address space is different. This adds a ton of overhead in a bunch of different places. For example when doing a parallel hash join, Postgres will have each worker build a local hash table. Then it will take all…

Passing pointers is not significantly faster than passing offsets into a shared memory pool.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#513

Post by ikari_pl »

sevg wrote: post #509
mulmen wrote: post #434

Earlier quoted context omitted.

I performed both searches on Kagi and didn't see anything about IBM. I do see results for "DataBlade" and "Data blade database" but I didn't try those specific variations after my first two attempts returned nothing of interest. That's too much effort to decipher a HN post.

> That's too much effort to decipher a HN post. It would have been less effort than it took you to write this comment. Perhaps next time that you can’t be bothered, just ignore the comment and move on to another thread that meets your required spoonfeeding levels.

ignoring the part where you tell others how to live, I believe the effort estimates you used suggest you may struggle typing and project it on others.

Doing a few searches and reviewing their results is effort.

Google doesn't really say what went wrong if it's about Informix.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#514

Post by skybrian »

dimes wrote: post #415
malisper wrote: post #258

Earlier quoted context omitted.

Can you elaborate on the use case for query multiplexing? Is it so your client would only need to establish one connection with Postgres and then could run as many queries as it wanted?

Multiplexing would have a number of benefits. As you say, each client would only need a single connection regardless of the number of queries being sent. Resulting in: On the client side, there is usually a local connection pool. When a burst of traffic comes in, the client needs to either wait for the pool to free up or establish a new connection, which adds latency. This latency hit wouldn’t occur with multiplexing…

At that point does it make sense to use http/3 to avoid head-of-line blocking on the network?

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#515

Post by egorfine »

malisper wrote: post #147

Hey author here. Wasn't expecting to see this up. To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture. I'm now working on a new, not yet published version of pgrust that inc…

> build a better version of Postgres

Faster is quantifiable. How do you measure better?

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#516

Post by srdjanr »

ben0x539 wrote: post #494
Chyzwar wrote: post #203

I think the best way to test this would be to put PgBouncer or a similar proxy in front of a busy production database, and mirror queries to both traditional Postgres and the Rust one at the same time. Then you can compare output and performance under real load. After running it for a while, you could diff the tables one to one against the normal Postgres instance.

Can you control the timing of queries across two db instances well enough to expect the tables to be identical?

Good question, I guess you can run two identical Postgress instances and check if they have a diff

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#517

Post by egorfine »

Blahagun wrote: post #483
sdevonoes wrote: post #251

Don’t understand these rewrites. - typically they are behind a single person. That’s usually bad because of spf - typically they are achieved in a very short amount of time, so the author hasn’t acquired any discipline in creating the project. That means it’s unlikely the author is going to stick to the project in the mid and long term - anyone that wants to contribute to the project needs to pay. Needs to pay tokens…

It's a very interesting phenomenon in recent years and most of the discussions about "why" are immediately blocked by the "memory safety" argument, as if it's a silver bullet for all things that are considered "bad" in software implementations. No matter how good the language is, and I consider Rust a very good language, it's practically impossible to replace years of experience and tested code, most of it contribute…

> impossible to replace years of experience

People with no experience don't have enough experience to realize what exactly they are missing here.

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#518

Post by cyphar »

IsTom wrote: post #339
rjh29 wrote: post #282

Earlier quoted context omitted.

AI is a great use for this kind of boring, rote translation where precision is important. Humans are quite bad at it and tend to make mistakes. In either case the focus should be on improving testing, not trying to manually verify if the translation was correct by eye.

With programs large enough tests aren't going to ever be enough. Formal verification might work, but then who checks the specification for bugs?

I really wonder where all of these people who believe that tests perfectly encapsulate the behaviour of software come from. Maybe it's because LLMs happen to work better when you give them acceptance criteria and people struggle to distinguish between "better" and "good"?

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#519

Post by guenthert »

frollogaston wrote: post #364
dupontcyborg wrote: post #354

Earlier quoted context omitted.

i may be missing context, but shared memory across processes, without ipc?

Being really pedantic here, shared memory is considered IPC, but not the kind you're thinking of. Shared address space, no overhead.

As long as we're pedantic ... the subject is shared memory. Unless you specify the same, non-null, target address in the call to mmap (and the kernel happens to grant you that mapping on all calling sites), the addresses will be different; the address space is not shared (each mapping might also have different access permissions).

That distinction is important as pointers generally cannot be shared (a problem which can of course be solved with one more indirection ;-) .

Re: Postgres rewritten in Rust, now passing 100% of the Postgres regression tests

#520

Post by egorfine »

Blahagun wrote: post #483
sdevonoes wrote: post #251

Don’t understand these rewrites. - typically they are behind a single person. That’s usually bad because of spf - typically they are achieved in a very short amount of time, so the author hasn’t acquired any discipline in creating the project. That means it’s unlikely the author is going to stick to the project in the mid and long term - anyone that wants to contribute to the project needs to pay. Needs to pay tokens…

It's a very interesting phenomenon in recent years and most of the discussions about "why" are immediately blocked by the "memory safety" argument, as if it's a silver bullet for all things that are considered "bad" in software implementations. No matter how good the language is, and I consider Rust a very good language, it's practically impossible to replace years of experience and tested code, most of it contribute…

> it's an attempt to hijack popular software projects

Exactly. And this is why those projects are typically called "XXX in Rust" or "XXX-rs". Because the creators get to do their favorite thing - coding in their loved language - while skipping all the hardships of designing, accepting real feedback, involving users and getting traction - all while simultaneously hijacking the existing brand.

As of now I know of a single project that changed their name after being called and the project surprisingly got some traction.

Post reply on HN