Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Preview of DuckDB v2.0

duckdb.org

Post reply on HN
Print view View original

111–120 of 144 posts

Re: A Preview of DuckDB v2.0

#111

Post by Mic92 »

nh2 wrote: post #95
therealdrag0 wrote: post #81

Hate to bring it up, but 10,000 commits in less than 6 months is a lot. Is AI a major contribute here? Is AI use for accelerated development of a beloved tool like DuckDB enough to quiet lingering doubters?

If you merge PRs that have commit mesages like this, it's easy to arrive at 10000 commits in 6 months: rename to NodePointer instead format Revert "format" Revert "rename to NodePointer instead" rename to OptionalNodePtr woops update comment slot renames more renames Source: https://github.com/duckdb/duckdb/pull/23605 If every Ctrl+S is a commit, it'll go up fast. "woops"!

They seem to squash PRs using the pr description as commit message. So this sounds like a good approach to me.

Re: A Preview of DuckDB v2.0

#112

Post by jinjin2 »

I’m mostly using Exasol these days (the concurrency and smooth scaling to multi-node is just too seductive), but with the introduction of Quack I might take another look at DuckDB. I’ll have to see how well it handles many agents reading and writing to it concurrently.

Re: A Preview of DuckDB v2.0

#114

Post by AlfeG »

throwaw12 wrote: post #9
jtbaker wrote: post #8

DuckDB is one of the things I've been most excited about in a long time. Introduced it to projects at 3 companies since 2023, greatly lowering resource requirements and running it in a variety of environments. Just having the ability to do out of core bigger than memory data processing on lower end consumer grade hardware is remarkable. Thanks to the team for everything!

Curious to learn more about how people are using it? Are they downloading parquet files and running analyses locally, or are they connecting to Iceberg-like data lake and leveraging DuckDBs query engine capabilities or have you exposed an interface (REST, UI) to query your data?

Realtime full MSSQL database mirroring into DuckDb to do a complex reporting. Everything is in-process. DuckDb database mapped to temp storage and recreated on app restart. Still order of magnitude faster then doing a direct query over MSSQL Server (2ms vs 40+ seconds on same query).

Some devs in team still cannot believe that there is no cheating, that it's possibe, that some 60Mb DB can do queries faster then MSSQL Server with just around 250Mb+ of memory overhead.

(.Net 10 + DuckDB.NET package)

Re: A Preview of DuckDB v2.0

#115

Post by mastermage »

I love DuckDB genuinely more than sqlite even though they do completely different things but DuckDB has like for me the perfect mix between simplicity, embedded capabilities and expressiveness. (and actual Time and Date Types).

Re: A Preview of DuckDB v2.0

#116

Post by mastermage »

nh2 wrote: post #95
therealdrag0 wrote: post #81

Hate to bring it up, but 10,000 commits in less than 6 months is a lot. Is AI a major contribute here? Is AI use for accelerated development of a beloved tool like DuckDB enough to quiet lingering doubters?

If you merge PRs that have commit mesages like this, it's easy to arrive at 10000 commits in 6 months: rename to NodePointer instead format Revert "format" Revert "rename to NodePointer instead" rename to OptionalNodePtr woops update comment slot renames more renames Source: https://github.com/duckdb/duckdb/pull/23605 If every Ctrl+S is a commit, it'll go up fast. "woops"!

i love woops

Re: A Preview of DuckDB v2.0

#117

Post by hantusk »

dangoodmanUT wrote: post #89

It's funny to me that we still don't have incremental materialized views. All of the parts are there (export state, agg_state (forget fn name), finalize). I wonder if they're avoiding an explicit war with clickhouse or something. I do recall they mentioned they want to add this to ducklake. Incremental MVs are ClickHouse's best feature. If DDB adds this, the last moat is distributed query execution.

It is implemented here as DuckDB extension: https://github.com/ila/openivm/

Re: A Preview of DuckDB v2.0

#118

Post by MILP »

Love DuckDB. It’s so fast and portable! I mainly use the query engine as part of my ETL process for creating SQLite database. I’d love to pitch it at work, but we’re heavily invested in BigQuery, which makes it a bit difficult.

Re: A Preview of DuckDB v2.0

#120

Post by rubenfiszel »

We have bet early on DuckDB and Ducklake for Windmill and couldn't be happier. The focus on server/client mode is interesting, it opens the way for orchestrators like ours to have "lite" workers/jobs for duckdb that connect to one central bigger beefy nodes and improve the overall efficieny. I'm very curious if benchmark shows that there are performance benefits to do so thanks to co-location and overall less cpu cycles wastes.
Post reply on HN