Heartening to see so many "This is an apple, that is an orange" comments. Spot on folks!
Choose DuckDB rather than SQLite
41–50 of 65 posts
Re: Choose DuckDB rather than SQLite
#42I couldn't read the article as it read like AI.
And I am fatigued by the AI style in all code comments, reviews, PRs etc :(
Re: Choose DuckDB rather than SQLite
#43As many others have said here, you should just think about transactional vs analytical as well as single user in-process vs multi-user client-server when making choices.
That said, I do think duckdb has a wider range of use cases than people here might think. It can whip through fairly large datasets (I use it for ~1B row tables all the time)
Re: Choose DuckDB rather than SQLite
#44The AI slop is tiring man wtf. It’s so fucking lazy. The benchmark is comparing apples to oranges and doesn’t seem to be aware of it, and the way it’s written just reeks of LLM.
Re: Choose DuckDB rather than SQLite
#45SQLite can be replicated. rqlite, dqlite, litestream etc
DuckDB can't be. PR was sent a year ago. Blocked on the same concurrency model issue in the other sub thread.
Specifically on windows, the database can't read its own WAL file from a different thread in the same process.
Love DuckDB for being permissively open source, great tech and performance!
Re: Choose DuckDB rather than SQLite
#46> DuckDB's columnar engine That is workload specific. Title should be "Choose DuckDB rather than SQLite for Analytics" IMHO
DuckDB also implements MVCC. In DuckLake, the DB is used for absorbing small writes and compacting them before sending to object storage.
It has some characteristics typical of OLTP engines. But they are targeted and limited to areas DuckDB feels are important.
Re: Choose DuckDB rather than SQLite
#47How about writing with sqlite and querying with duckdb-sqlite, using a replica or WAL, to avoid locks?
Re: Choose DuckDB rather than SQLite
#48Earlier quoted context omitted.
> SQLite is still the GOAT when it comes to OLTP, but DuckDB is really becoming the GOAT in the OLAP world Can you explain this more, especially why SQLite is best at OLTP and what happens at scale?
Most people forget that clickhouse embedded exists
Let's keep discussion bar high.
I've just checked their website, and they state "relying on ClickHouse to power these analytics use cases". That's not OLTP. https://clickhouse.com/comparison/postgresql
Fair to say, seeing 1000x w/o any trace of proof won't help me to choose.
Re: Choose DuckDB rather than SQLite
#49Despite its obvious advantages, the biggest drawback of DuckDB is its concurrency model [1]. If a process opens a database in read-write mode, it acquires an exclusive lock on the file. This prevents even simple read operations from other processes as long as the writer remains open. Maybe there's a simple workaround I haven't come across, but I found it to be quite a productivity killer. So yes, all these benchmarks…
This is actually the biggest reason I don't automatically choose sqlite for small project databases. I've built a lot of toy utilities that crossed a bare threshold of usefulness, and then suddenly it's not a toy anymore when people start using it.
Now it's debt. Oops.
Re: Choose DuckDB rather than SQLite
#50Earlier quoted context omitted.
Most people forget that clickhouse embedded exists
Let's keep discussion bar high. I've just checked their website, and they state "relying on ClickHouse to power these analytics use cases". That's not OLTP. https://clickhouse.com/comparison/postgresql Fair to say, seeing 1000x w/o any trace of proof won't help me to choose.
You new to the world, right? ClickHouse proven being a beast for analytics, tracing and even some basic metrics storage (VM won of course, but still CH is probably 2nd or 3rd)