31–40 of 66 posts
Earlier quoted context omitted.
For example you cannot have concurrent access. As soon as you need a worker process and a web process SQLite is out. Or if you are trying to use it as a vector db all of those vector searches will block a node event loop.
https://lobste.rs/s/ko1ji1/lobste_rs_is_now_running_on_sqlit...
And what do you do when you need more than one server, for redundancy or scaling to more servers?
Overall premise is wrong though. Moving the database out of process will change performance characteristics and data architecture too much and will cause massive headaches at exactly the time when you are trying to scale with success. You should have out of process performance tests early to catch these issues, even if you do deploy a single node.
If success can be satisfied with a single node and you are satisfied with availability and recovery that gives you then great, but it isn't all I need.
Earlier quoted context omitted.
> SQLite is all you need if it really is all you need. most use cases are really capable of being satisfied by sqlite, but the "architect" imagines they need more (or is preparing for the potential).
> the "architect" imagines they need more (or is preparing for the potential) I guess I am one of those "architects" that imagines they need an actual date/time storage class instead of some stringly-typed text column that I hope will contain a parsable ISO8601 datetime string when I try to read it back. Hipp said that it will never be added because it will bloat the size of the embedded object. Because that is what…
Author here. This started because I read Evan Hahn's STRICT tables post [1] last week and got curious how far "just use SQLite" actually holds up under real load, not toy benchmarks. So I built a small social app (Chirp: 50k users, 1M posts, ~2.5M follows) in one SQLite file, put it behind a plain Node server, and load tested it properly: real HTTP, real JSON serialization, autocannon hammering it over sockets. The w…
Your later advice about "VACUUM INTO (backupfile)" is good, though: the SQLite manual guarantees that that's safe. But it's not safe to back up with `cp` if there are transactions currently writing to the DB: that has a chance of copying files in an inconsistent state, resulting in a corrupt DB and data loss you don't realize has happened until you restore the backup and find out there are some rows missing, or some rows have an invalid mix of old and new data.
Earlier quoted context omitted.
How are lobster.rs handling writes? Are they queuing them synchronously?
https://sqlite.org/wal.html presumably.
So lunchtime in America there will be multiple people posting comments at the same time
And Sqllite only allows one writer, so what happens if multiple people send their comments at once
So are they using ultra fast storage on the host or queues or other
Earlier quoted context omitted.
> the "architect" imagines they need more (or is preparing for the potential) I guess I am one of those "architects" that imagines they need an actual date/time storage class instead of some stringly-typed text column that I hope will contain a parsable ISO8601 datetime string when I try to read it back. Hipp said that it will never be added because it will bloat the size of the embedded object. Because that is what…
In fairness, sqlite is perfectly happy with Julian dates or Unix timestamps (that’s the affinity of a column typed “datetime” in non-strict mode) and timestamp(tz) are nothing to write home about except in complaint.