Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Preview of DuckDB v2.0

duckdb.org

Post reply on HN
Print view View original

31–40 of 144 posts

Re: A Preview of DuckDB v2.0

#31

Post by jtbaker »

fifilura wrote: post #20
jtbaker wrote: post #8

DuckDB is one of the things I've been most excited about in a long time. Introduced it to projects at 3 companies since 2023, greatly lowering resource requirements and running it in a variety of environments. Just having the ability to do out of core bigger than memory data processing on lower end consumer grade hardware is remarkable. Thanks to the team for everything!

My favourite is AWS Athena (backed by Trino). "If we use this we get indefinite RAM indefinite CPU and do not need to host a server". I had an impression that DuckDB was not great at distributing work to other machines, but good at doing it locally? Am I wrong?

I don't think DuckDB itself can coordinate work across multiple nodes. But you could put it behind an HTTP layer and scale horizontally based on resource utilization?

Re: A Preview of DuckDB v2.0

#32

Post by otter-in-a-suit »

Super excited about Quack (partially due to the name). I use duckdb for both analytics and runtime, but I do have to serve/handle/manage a giant, multi-GiB duckdb file as effectively a runtime artifact[1]. I'm aware that this isn't the _perfect_ database for this, but the mix of it being fast, having spatial support, sane coding interfaces, great dbt integration, and me being able to do everything between "run a giant several hundred step dbt pipeline" to "query the output of said pipeline" to "read/query a csv on disk" with the exact same tool is just so nice. If I could centrally manage said asset more akin to a traditional database, I'd be very happy.

I've partially solved this with separate databases for different steps in the data pipeline(s) and have even experimented with Clickhouse as a complete alternative, but I really like way too many things about duckdb to replace it.

[1]: If you care: https://skaldmaps.com/blog/2026/07/zip-codes-are-a-bad-spati...

Re: A Preview of DuckDB v2.0

#33

Post by drannex »

Really looking forward to that new Async system, especially when reading/querying against thousands of parquet files. This is going to monumentally affect me and my work - I have to query against millions of massive parquet files and the speed has already been rather wonderful, but if those metrics are to be even 100% in range, this is going to make life so much better.

DuckDB is seriously an incredible utility.

Re: A Preview of DuckDB v2.0

#35

Post by drannex »

badatnames wrote: post #22

Looks like an awesome release, but the smell of AI from that post is horrid. Here is a wild idea: is it really so hard to edit out sentences structured and punctuated like this - it's so painfully obvious and distracts from the content. The effect is real.

I don't really get an AI smell on this, in fact I see multiple parts that an AI would have corrected - grammatical issues, personal writing quirks, etc.

Writing similar to this: is quite a normal way of writing for technical articles - especially when you are trying to make the point clear and well organized.

Re: A Preview of DuckDB v2.0

#36

Post by mediaman »

otter-in-a-suit wrote: post #32

Super excited about Quack (partially due to the name). I use duckdb for both analytics and runtime, but I do have to serve/handle/manage a giant, multi-GiB duckdb file as effectively a runtime artifact[1]. I'm aware that this isn't the _perfect_ database for this, but the mix of it being fast, having spatial support, sane coding interfaces, great dbt integration, and me being able to do everything between "run a gian…

I built a platform for some midsize companies in a specific vertical that is basically a data warehouse with some LLM-driven dashboarding and query tools on top. Typical data size 5-150gb. So I built a service layer around duckdb, where each tenant gets their own duckdb. I'm also in the boat of knowing that duckdb is not the perfect solution for this (the classic use case is running it against local data on a laptop), but there's so much I like about it, and it's really nice that each tenant can have total separation with custom schemas and that it's straightforward to pile data into object storage and form your own lake.

And now with things like quack the sharp points around concurrency are relaxing, and it feels like the compromises of using it this way are disappearing.

Re: A Preview of DuckDB v2.0

#38

Post by tuvix »

brunoborges wrote: post #37

How does DuckDB compares with PostgreSQL / MariaDB ?

DuckDB is an in-process (now I guess less so with Quack) OLAP database for analytical workloads. PostgreSQL or MariaDB are OLTP row-oriented databases that are great for application/transaction-focused workloads but are less great when you want to query across a giant amount of data.

Re: A Preview of DuckDB v2.0

#39

Post by peesem »

throwaw12 wrote: post #9
jtbaker wrote: post #8

DuckDB is one of the things I've been most excited about in a long time. Introduced it to projects at 3 companies since 2023, greatly lowering resource requirements and running it in a variety of environments. Just having the ability to do out of core bigger than memory data processing on lower end consumer grade hardware is remarkable. Thanks to the team for everything!

Curious to learn more about how people are using it? Are they downloading parquet files and running analyses locally, or are they connecting to Iceberg-like data lake and leveraging DuckDBs query engine capabilities or have you exposed an interface (REST, UI) to query your data?

maybe a niche use case but i've found it's perfect to store & query random trivia/gameshow questions based on filters for my personal clones of things like Family Feud and Jeopardy

Re: A Preview of DuckDB v2.0

#40

Post by tccole »

mediaman wrote: post #36
otter-in-a-suit wrote: post #32

Super excited about Quack (partially due to the name). I use duckdb for both analytics and runtime, but I do have to serve/handle/manage a giant, multi-GiB duckdb file as effectively a runtime artifact[1]. I'm aware that this isn't the _perfect_ database for this, but the mix of it being fast, having spatial support, sane coding interfaces, great dbt integration, and me being able to do everything between "run a gian…

I built a platform for some midsize companies in a specific vertical that is basically a data warehouse with some LLM-driven dashboarding and query tools on top. Typical data size 5-150gb. So I built a service layer around duckdb, where each tenant gets their own duckdb. I'm also in the boat of knowing that duckdb is not the perfect solution for this (the classic use case is running it against local data on a laptop)…

I feel like familiarity and ease of use and “good enough” beat out the perfect db for the job in many occasions.
Post reply on HN