DuckDB 0.8
duckdb.org
DuckDB 0.8
1–10 of 101 posts
Re: DuckDB 0.8
#2Damn. This database quacks. Huge list of awesome super helpful tools.
Tons of great features & speed for everyone. Also love the stuff from the edge, such as Arrow DataBase Connector support,
> From this release, DuckDB natively supports ADBC. We’re happy to be one of the first systems to offer native support, and DuckDB’s in-process design fits nicely with ADBC.
Re: DuckDB 0.8
#3What a great database.
This and sqlite-utils are my go to tools for ad hoc data wrangling.
Re: DuckDB 0.8
#4Lack of order-preserving parallel parquet writing was my biggest gripe with DuckDB. Glad to see they implemented it! Between this duckdb release, DataFusion and Apache Arrow (yes, I know that helps power the other projects I listed), I’m hoping I won’t have to touch a spark cluster for a long time.
Re: DuckDB 0.8
#5I've been thinking about rebuilding the website analytics service I have, to be around sharded duckdb and just giving people raw SQL access.
ClickHouse is the alternative, but this approach has some interesting advantages, like simplicity.
Re: DuckDB 0.8
#6I recently pulled DuckDB out of a project after hitting a memory corruption issue in regular usage. Upon investigating, they had an extremely long list of fuzzer-found issues. I just don't understand why someone would start something in a memory unsafe language these days. I cannot in good conscience put that on a customer's machine.
We ended up rewriting a component to drop support for Parquet and to just use SQLite instead. I love the idea of being able to query Parquet files locally and then just ship them up to S3 and continue to use them with something like Athena.
The other thing that rubbed me the wrong way was that rather than fix the issue, they just removed functionality. DuckDB (unironically) needs a rewrite in Rust or a lot more fuzzing hours before I come back to it. While SQLite is not written in a memory safe language, it is probably one of the most fuzzed targets in the world.
Re: DuckDB 0.8
#7This is the first time I heard about DuckDB. Why someone would use that instead of a mature engine like SQLite?
Re: DuckDB 0.8
#8This is the first time I heard about DuckDB. Why someone would use that instead of a mature engine like SQLite?
Columnar database not row based. More appropriate for analytical workloads, and not as a transactional backend.
Re: DuckDB 0.8
#9I recently pulled DuckDB out of a project after hitting a memory corruption issue in regular usage. Upon investigating, they had an extremely long list of fuzzer-found issues. I just don't understand why someone would start something in a memory unsafe language these days. I cannot in good conscience put that on a customer's machine. We ended up rewriting a component to drop support for Parquet and to just use SQLite…
>The other thing that rubbed me the wrong way was that rather than fix the issue, they just removed functionality.
It is a limited team size. If they feel a feature is causing too much grief, I would rather they drop it than post a, "Here be dragons" sign and let users pick up the pieces.
Edit: missed an obvious opportunity to take a shot at MySQL
Re: DuckDB 0.8
#10I recently pulled DuckDB out of a project after hitting a memory corruption issue in regular usage. Upon investigating, they had an extremely long list of fuzzer-found issues. I just don't understand why someone would start something in a memory unsafe language these days. I cannot in good conscience put that on a customer's machine. We ended up rewriting a component to drop support for Parquet and to just use SQLite…
Sorry to hear that!
(I work on docs for the DuckDB Foundation)
Starting in this release, the DuckDB team invested significantly in adding memory safety throughout catalog operations. There is more on the roadmap, but I would expect this release and all following to have improved stability!
That said, at my primary company, we have used it in production for years now with great success!