Viewing profile — lnkuiper

Stabilizing storage is high priority for us! You won’t notice anything new while using DuckDB, but we’ve actually rewritten our entire serialization framework in this release. This…

We plan to update our results on ClickBench at some point, as we’ve made some improvements there in this release. I’d expect performance to be similar

This is very true. DuckDB does not support JSON because it’s a good tabular format, but because JSON is ubiquitous, and there are many use cases where querying JSON dumps for analy…

DuckDB will use multiple threads for reading the same file. Each thread will read different parts of the file, but the output will be in the order that the file came in due to Duck…

There are plenty of TopN sorts in TPC-DS, but not many (if any) regular sorts, so no. There are order-dependent window functions in there though, which did show a speed up.

You are right, we could probably re-run SQLite purely in memory, but only because macos dynamically allocates additional swap. However, I would not expect much improved performance…

Exactly. Even when it’s inlined, having a comparator with if/else is much, much slower than a single memcmp

Customer fits in memory, whereas catalog_sales does not. We chose to remove SQLite from the results because it was so much slower. The plots are much less readable when they are st…

I'm not sure if I understand, but we did not use a memory map (mmap), but rather blocks of memory that are explicitly (un)loaded by the buffer manager. The M1 + SSD performs really…

We wanted to use the same setup for all experiments, so we had to choose for an on-disk DB for SQLite, because TPC-DS SF100 catalog_sales does not fit in 16GB memory.