31–36 of 36 posts
This library is wild https://github.com/cvilsmeier/sqinn Sqlite over stdin, to a subprocess, and it's fast!
That's an interesting approach but doesn't it mean that if you want multiple connections at the same time, you'd need multiple subprocesses? Perhaps I misunderstand though.
Excellent evaluation. From reading the code, it appears that the units for the numbers column is usually milliseconds (ms) It also looks like squinn is the clear leader for most but not all of the benchmarks. Even though it's "not scientific", is still very useful as a baseline - thanks for taking this effort and publishing your results! Also taking a look at monibot.io , looks cool
Nit: "For benchmarks I used the following libraries: ". This is begging to be a table.
For a project a while back, I needed to turn many-gigabyte Postgres CSV table dumps into SQLite databases. I turned to Go as its a great language for easy parallelism combined with enough memory layout control to get relatively good performance. I quickly ruled out using database/sql drivers as the indirection through interface types added a bunch of overhead and stymied my attempts for reasonable memory layout. For…
There may be an opportunity to switch from TRANSIENT to STATIC in sqinn but I didn't read deeply enough to follow what the current memory approach is. https://github.com/cvilsmeier/sqinn/blob/a88b3df6c89f0531e39...
Earlier quoted context omitted.
There may be an opportunity to switch from TRANSIENT to STATIC in sqinn but I didn't read deeply enough to follow what the current memory approach is. https://github.com/cvilsmeier/sqinn/blob/a88b3df6c89f0531e39...
According to my private tests, using STATIC instead of TRANSIENT does not yield a significant performance boost (maybe 1 to 5 percent, if at all).
Earlier quoted context omitted.
Sure but once you have WAL logs, you suddenly have a more heavy weight setup. Backing it up you'll want to back up those WAL logs to achieve proper point in time recovery, and so on. My point is, you're now bolting on extra stuff on it to do things that Postgres can do (which can be pretty light weight). Not disrespecting SQLite, still one of my favorite DB's.
What? Why are you backing up the WAL? sqlite3 source_database.db ".backup backup_database.db" Now the WAL content is rolled into your new backup file. Stick a timestamp in the backup file name and run this as a cron job every N minutes and you have all the recovery you need. Another one-liner to sync to S3 and you're all set. Edit: And just to clarify, that command can be run on a live DB as it's being used by your a…