I love reading all of the DuckDB love here. It's good software.
A Preview of DuckDB v2.0
131–140 of 144 posts
Re: A Preview of DuckDB v2.0
#132Earlier quoted context omitted.
Wait, what? I‘ve always assumed MotherDuck is the company by the main developers of DuckDB? They are unrelated?
They launched ducklake: https://ducklake.select/
Yeah, that was one of the sources of my confusion, too.
Some of the recent DuckDB features and extensions would certainly seem to undercut premium features MotherDuck has layered over DuckDB. Though, on the flip side, since there is a relationship between the orgs it does seem like MotherDuck features are a good way for the DuckDB team to field test (future) features without committing to building extensions or adding them to the database itself (e.g. DuckLake and Quack).
It is all a bit curious, though, and makes you wonder what's in store for the future. The DuckDB team has hinted at some "big things" coming soon which I don't think are covered in these release notes.
Re: A Preview of DuckDB v2.0
#133The single-writer limit was always the awkward part. Quack finally addresses it.
Re: A Preview of DuckDB v2.0
#134[dead]
Re: A Preview of DuckDB v2.0
#135That feature set is crazy! I absolutely love what you guys did. Thanks for the hard work!
Re: A Preview of DuckDB v2.0
#136I <3 DuckDB. It has become one of my go to tools for storing, data processing , integrations and now even graph. More importantly it's fun to use because it is so portable. Looking forward to v2.
What advantages does it have over SQLite in your use cases? Can you give any examples?
DuckDB is heavily focused on analytical workflows, so it stores data in a compressed columnar format. So "rewrite one row" is more efficient in SQLite, but "sum all the values in this column" is vastly more efficient in DuckDB.
That's the core of the difference, to me. But the projects also have a pretty different ethos. SQLite has always prioritized simplicity, often at the expense of features, whereas DuckDB's approach to features seems to be "the more the merrier." E.g. the star feature of this new release, client/server mode. Plenty of people would love to see something similar in SQLite, but it doesn't seem likely.
Re: A Preview of DuckDB v2.0
#137Earlier quoted context omitted.
Curious to learn more about how people are using it? Are they downloading parquet files and running analyses locally, or are they connecting to Iceberg-like data lake and leveraging DuckDBs query engine capabilities or have you exposed an interface (REST, UI) to query your data?
Realtime full MSSQL database mirroring into DuckDb to do a complex reporting. Everything is in-process. DuckDb database mapped to temp storage and recreated on app restart. Still order of magnitude faster then doing a direct query over MSSQL Server (2ms vs 40+ seconds on same query). Some devs in team still cannot believe that there is no cheating, that it's possibe, that some 60Mb DB can do queries faster then MSSQL…
how are you doing the etl from mssql to duckdb? are you doing naive snapshots? or ducklake?
Re: A Preview of DuckDB v2.0
#138> The VARIANT type shipped in DuckDB v1.5, and the way to think about it is JSON on steroids. Basically, imagine if JSON were fast. [...] DuckDB automatically detects the common structure hidden in your semi-structured data and “shreds” it, so it compresses well in storage I am really looking forward to this hitting v2.0. I can't stand uncompressed JSON - so space-inefficient. But heterogenous JSON in parquet files i…
i know they're not making any promises but i'm also very excited for it
Re: A Preview of DuckDB v2.0
#139Super excited about Quack (partially due to the name). I use duckdb for both analytics and runtime, but I do have to serve/handle/manage a giant, multi-GiB duckdb file as effectively a runtime artifact[1]. I'm aware that this isn't the _perfect_ database for this, but the mix of it being fast, having spatial support, sane coding interfaces, great dbt integration, and me being able to do everything between "run a gian…
[flagged]
Re: A Preview of DuckDB v2.0
#140Earlier quoted context omitted.
UPDATE: Removed uninformed comment about the relationship between DuckDB and MotherDuck.
Wait, what? I‘ve always assumed MotherDuck is the company by the main developers of DuckDB? They are unrelated?
Ryan Boyd, co-founder at MotherDuck here. Yep, we've worked closely with the team, but are separate companies! We're focused on cloud data warehousing and customer-facing analytics. Let me know if you have any other questions!