It is not really Postgres, the queries are run on DuckDB. Yeah, DuckDB is really fast for analytical queries.
Yes but this is an extension that enables abstracting that away for an end user and allows them to just use psql
Postgres Just Cracked the Top Fastest Databases for Analytics
21–30 of 126 posts
Re: Postgres Just Cracked the Top Fastest Databases for Analytics
#22> To enhance query execution speed, we embedded DuckDB as the execution engine for columnstore queries So is it Postgres or DuckDB that cracked the analytics top ?
well, pg_mooncake is a Postgres extension, and Postgres + pg_mooncake is still just Postgres. Users deploy pg_mooncake as a Postgres extension and write and query all tables through psql. Fast analytic databases need two key things: columnar storage and a vectorized execution engine. We introduce a columnstore table access method in Postgres with data stored in Parquet) and execute queries on those tables using DuckD…
One thing I forgot to mention. We're actually faster than DuckDB on parquet. We implemented segment elimination based on parquet metadata.
We blogged about this: https://www.mooncake.dev/blog/duckdb-parquet
Re: Postgres Just Cracked the Top Fastest Databases for Analytics
#23Sai from PeerDB/ClickHouse here. Nice to see the progress on this project! I wanted to leave a few notes:
For analytics on transactional data, it looks like you'd still need to use logical replication (https://github.com/Mooncake-Labs/pg_mooncake/issues/90). Logical replication is somewhat similar to an ETL/CDC experience, though it's more Postgres-native. Managing logical replication at a production grade isn't trivial — it's quite common for customers to use PeerDB for homogeneous replication due to performance, manageability and observability issues with logical replication.
One potential value of extensions is Postgres compatibility for your queries. However, I'm curious about the SQL coverage on columnar tables — for example, advanced constructs like partitioned tables, prepared statements, correlated subqueries, RCTEs, triggers, and more. While it seems there’s ongoing work to improve compatibility, achieving full Postgres compatibility is challenging. In some cases, you might need to fall back to Postgres (instead of DuckDB), sacrificing performance for certain queries.
The biggest challenge we faced at Citus was the constant trade-off between Postgres compatibility and performance — both are complex and constantly moving targets. This was the key reason why Citus couldn't compete with purpose-built databases like Snowflake, ClickHouse, SingleStore. While DuckDB didn’t exist when we built Citus, it's still fundamentally different from Postgres. Even though the DuckDB dialect is somewhat similar to Postgres (as is Snowflake’s), retrofitting two distinct databases — each evolving independently for world's best performance — isn't trivial.
In short, relying on ETL (logical replication) without providing full Postgres compatibility raises the question: is it better to lean on a purpose-built database with a laser focus on performance, or adopt a more generalized approach?
Anyway, I come from a different school of thought — using the right tool for the right job. That said, I love seeing all the progress and evolution in the Postgres community — Postgres will always be my first love!
Re: Postgres Just Cracked the Top Fastest Databases for Analytics
#24Congrats on the launch!
Does it support ordinary db migrations with the normal Postgres interface?
Re: Postgres Just Cracked the Top Fastest Databases for Analytics
#25They're already working on 0.2 https://www.mooncake.dev/blog/pgmooncake-v02discussion
oh yeah. We're baking 0.2. Extremely excited about the small-writes.
Today, each insert will create a new parquet file.
This will be fixed in v0.2. Stay tuned, we will have some announcements here coming soon.
Re: Postgres Just Cracked the Top Fastest Databases for Analytics
#26> To enhance query execution speed, we embedded DuckDB as the execution engine for columnstore queries So is it Postgres or DuckDB that cracked the analytics top ?
well, pg_mooncake is a Postgres extension, and Postgres + pg_mooncake is still just Postgres. Users deploy pg_mooncake as a Postgres extension and write and query all tables through psql. Fast analytic databases need two key things: columnar storage and a vectorized execution engine. We introduce a columnstore table access method in Postgres with data stored in Parquet) and execute queries on those tables using DuckD…
What is the cost of scaling up number of CPUs for parallel processing?
That was always the culprit for me compared to AWS/Athena and BigQuery. They are dirt cheap on analytics workloads when you can parallelize the calculations to 100 CPUs without really paying any extra.
With postgres you are stuck with linear cost for scaling up number of CPUs, so everything is slow anyway.
Re: Postgres Just Cracked the Top Fastest Databases for Analytics
#27As others have mentioned the results are from an extension, not postgres.
Unfortunately using an extension that's not "part" of postgres (like pg_stat_statements) is not trivial for most people since there are both technical and legal issues.
Re: Postgres Just Cracked the Top Fastest Databases for Analytics
#28Looking at the effort being put in it seems there’s a massive market for proper Postgres analytics / columnstore, which is very telling of the pain involved in moving data between systems.
Yes this has been my experience too. It isn’t just the costs and complexities involved in the moving of the data, but needing the expertise in your team to do so or at least investing the time of someone in your team to figure it out and maintain it. All the while they can’t do anything that is actually useful.
Re: Postgres Just Cracked the Top Fastest Databases for Analytics
#29All you need is postgres: part 73
Or all you need is a database that allows anyone to skip the hard part and simply write a plugin for.
Re: Postgres Just Cracked the Top Fastest Databases for Analytics
#30Earlier quoted context omitted.
well, pg_mooncake is a Postgres extension, and Postgres + pg_mooncake is still just Postgres. Users deploy pg_mooncake as a Postgres extension and write and query all tables through psql. Fast analytic databases need two key things: columnar storage and a vectorized execution engine. We introduce a columnstore table access method in Postgres with data stored in Parquet) and execute queries on those tables using DuckD…
One thing I forgot to mention. We're actually faster than DuckDB on parquet. We implemented segment elimination based on parquet metadata. We blogged about this: https://www.mooncake.dev/blog/duckdb-parquet
Have you upstreamed the change to DuckDB?