Live data from Hacker News

SQLite: Past, Present, and Future

vldb.org

61–70 of 147 posts

Re: SQLite: Past, Present, and Future

#61

Earlier quoted context omitted.

No durability guarantee is a showstopper for any serious use case

If it's good enough for avionics and nuclear subs, it's probably good enough for most web apps.

Web apps do more concurrent writes than subs, plus you can configure SQLite for more durability

Re: SQLite: Past, Present, and Future

#62

Earlier quoted context omitted.

Not sure what you mean by durability. Sqlite has WAL that can be replicated (see litestream)

https://en.m.wikipedia.org/wiki/Durability_(database_systems...

https://www.sqlite.org/atomiccommit.html

sqlite is as good at durability as any non-replicated database, though you can configure it to be non-durable (most other databases too tbf).

https://www.sqlite.org/pragma.html#pragma_synchronous

By default WAL mode can rollback committed transactions in cases of power failure, but you can do `PRAGMA synchronous = FULL` to trade speed for durability.

Re: SQLite: Past, Present, and Future

#63
post #8
post #2

SQLite vs Postgres for a local database (on disk, not over the network): who wins? (Each in their most performance oriented configuration)

Postgres obviously. Sorry, just thought I'd buck the trend and assume a very write-heavy workload with like 64 cores. If you don't have significant write contention, SQLite every time.

If you can have one "database" thread and 63 "worker" threads, send messages back and forth, and don't hold open transactions, this would probably work with sqlite. Aka treat sqlite like redis.

Re: SQLite: Past, Present, and Future

#64

Earlier quoted context omitted.

you could make one pretty easily, no?

I'd like to see that. I also think the single write situation is not great for web applications, but I don't see an easy way around it without sacrificing things like consistency

If you can treat sqlite transactions like redis transactions (send the entire transaction up front) it can work.

Re: SQLite: Past, Present, and Future

#65
post #17

Earlier quoted context omitted.

Where is write contention coming from if it's operated locally?

... you can get tons of requests on a server?

Redis has the same limitation (only one transaction at a time) and is used a lot for webapps. It solves this by requiring full transactions up front. The ideal case for sqlite for performance is to have only a single process/thread directly interacting with the database and having other process/threads send messages to and from the database process.

Re: SQLite: Past, Present, and Future

#66

Earlier quoted context omitted.

https://en.m.wikipedia.org/wiki/Durability_(database_systems...

https://www.sqlite.org/atomiccommit.html sqlite is as good at durability as any non-replicated database, though you can configure it to be non-durable (most other databases too tbf). https://www.sqlite.org/pragma.html#pragma_synchronous By default WAL mode can rollback committed transactions in cases of power failure, but you can do `PRAGMA synchronous = FULL` to trade speed for durability.

I’m talking about the post I originally commented on. Things were disabled so durability is not guaranteed.

Re: SQLite: Past, Present, and Future

#67
post #42

I shared some notes on this on my blog, because I'm guessing a lot of people aren't quite invested enough to read through the whole paper: https://simonwillison.net/2022/Sep/1/sqlite-duckdb-paper/

I waited for a tl;dr but this is even better. Much appreciated.

Re: SQLite: Past, Present, and Future

#68

Earlier quoted context omitted.

If it's good enough for avionics and nuclear subs, it's probably good enough for most web apps.

Web apps do more concurrent writes than subs, plus you can configure SQLite for more durability

I don't have the data for subs, but there's web app and web app. No one is talking about using SQLite for 5k queries/s.

It might work, but I reckon 90% of web applications live beneath this relatively small threshold and 80% probably don't even reach 50 q/s.

Re: SQLite: Past, Present, and Future

#70

Earlier quoted context omitted.

https://www.sqlite.org/atomiccommit.html sqlite is as good at durability as any non-replicated database, though you can configure it to be non-durable (most other databases too tbf). https://www.sqlite.org/pragma.html#pragma_synchronous By default WAL mode can rollback committed transactions in cases of power failure, but you can do `PRAGMA synchronous = FULL` to trade speed for durability.

I’m talking about the post I originally commented on. Things were disabled so durability is not guaranteed.

Yeah that's not great
Post reply on HN