300x faster is nice, but I mostly clicked to see if operator fusion finally explains why my GROUP BY still feels like it is doing the work by hand.
Making Postgres 300x faster for analytics: batching, operator fusion, and SIMD
91–100 of 168 posts
Re: Making Postgres 300x faster for analytics: batching, operator fusion, and SIMD
#92pgrust looks interesting. Could it be used as a library by someone who wants a new DB for each integration test in their Rust test suite?
One of the new features we recently built is "test mode". This brings cloning a template db from 100ms down to If you're interested in trying it out, please reach out to me at malis@pgrust.com
Re: Making Postgres 300x faster for analytics: batching, operator fusion, and SIMD
#93Earlier quoted context omitted.
They could simply spend a few months and a few million tokens and get their own port, no? I doubt even 30000x faster would prompt a policy change.
No. This is system code. You let an LLM loose on it, it probably fixed 20 bugs and introduces 200 more plus 5 different performance regressions. In all fairness, your average app programmer would have the same problems. That's why it takes so long to learn to be a system programmer and why it takes so long to do anything on a systems source base. For reference, systems are OSes, DBs and compilers (although compilers…
What do you mean, "No"? Letting an LLM loose on Postgres is exactly what happened here: https://malisper.me/pgrust-rebuilding-postgres-in-rust-with-...
Re: Making Postgres 300x faster for analytics: batching, operator fusion, and SIMD
#94Question, does having it in pure rust, opens possibility of embedding pgrust directly into binary, making it an alternative to SQLite/turso?
Re: Making Postgres 300x faster for analytics: batching, operator fusion, and SIMD
#95Earlier quoted context omitted.
We disabled parallelism in the blog post for demonstration purposes. The 300x slower refers to the clickbench numbers[0] where parallelism is enabled [0] https://benchmark.clickhouse.com/#system=+liH|pgrs|gQ&type=-...
Who is "we"?
The pgrust team asked me to look at their results on a review system, and I confirmed the ClickBench speedup there. Regular PostgreSQL is really terrible at some of these queries. Unfortunately fixing that is hard to do in core itself because columnar storage lives outside of the main tree, and some optimization problems only show up when layered on columnar.
Re: Making Postgres 300x faster for analytics: batching, operator fusion, and SIMD
#96Thanks to the authors for choosing a license that respect users freedom, on top of being an awesome technical project.
While I do like pgrust's license, I do feel like it's kind of wrong to port in such a direct way and change the license. I guess this isn't a fork, but it kind of is? It looks like according to this post [0], they did a Claude Fable + Opus re-write. I know that legally this is seemingly a valid way to do things and avoid copyright, but it feels wrong to me. I don't even necessarily think that my feelings are correct,…
My understanding is that they used c2rust, and then told the LLM/Agents to make the code more idiomatic rust, while also using the PG test suite as a feedback mechanism. This is almost certainly a derived work (and thus a fork, and should thus have the original license and copyright preserved).
For example, if I compiled PG into x86-64 assembly, and then decompiled it into C (via, say, IDA), and then polished that decompiled C code into very readable C code, it is still a derived work. For some reason, people think that if you include an LLM or Agent, copyright can be ignored, and plagiarism is now no longer possible.
It is similar to the crypto-folks thinking that if you use crypto, you no longer have to pay taxes, because the internet/computers make all inconvenient realities go away.
Honestly, such flagrant and arrogant copyright violations make it hard for me to take the project seriously, because it seems like a desperate stunt for attention (which itself may be a solid business move, but that is besides the point).
Put differently, if one were to fork pgrust, strip away the new license and copyright, and restore the original PG license and copyright (while also adding malisper+team to that copyright), they would face no legal consequences at all. In fact, they would probably be a less legal risk than the pgrust team.
Re: Making Postgres 300x faster for analytics: batching, operator fusion, and SIMD
#97Earlier quoted context omitted.
If it is indeed 300* faster, I'm sure more rational corporations will rethink their license policy or be left in the dust.
License policies are made by lawyers not by programmers. And their competition will be in the exact same boat (different lawyers though). AGPL is so toxic that it tends to be checked for during M&A processes so even if the current batch of lawyers is ok with it there is a chance that a later batch of lawyers is not. Given that the target audience for this project is the larger companies you are going to end up with a…
A project can probably use the EUPL instead of the AGPL (EUPL is to AGPL, as MPLv2[0] is to GPL). Basically, EUPL is file-based, not project based, and so it is not aggressively viral. You can use the EUPL code any way you want, as long as you make the original code available, plus any modification to the original files.
[0]: With Exhibit B, which prevents relicensing to GPL. It is also analogous to CDDL.
Re: Making Postgres 300x faster for analytics: batching, operator fusion, and SIMD
#98I am very disappointed to see the direction:
It is moving from a "interesting attempt to recreate system software" to "building flashy but useless demo"
Everyone who knows a bit about databases knows the difference between execution models and what kind of optimization it brings.
Re: Making Postgres 300x faster for analytics: batching, operator fusion, and SIMD
#99So one trick you can do is make a ramfs/tmpfs and start Postgres on it.
You need a server with enough ram to fit it all.
But it kind of make the database fly.
Re: Making Postgres 300x faster for analytics: batching, operator fusion, and SIMD
#100Cool project but .. reality is that people will generally not choose pgrust over Postgres, even 5-10 years from now. The problem is not that it may be technically superior and faster by then, it's that it's not built by the trusted Postgres team. There's a lot more to trust than development velocity or performance. It's also about the longevity and continuity of a critical piece of technology.
There's also the fact that it's basically vibecoded.