SQLite improving performance with pre-sort
andersmurphy.com
SQLite improving performance with pre-sort
1–9 of 9 posts
Re: SQLite improving performance with pre-sort
#2Neat optimization, I've never seen that before. Thanks for sharing.
Re: SQLite improving performance with pre-sort
#3It's not clear in the code how you're batching the writes. Could the performance improvement be mostly explained by batches?
Re: SQLite improving performance with pre-sort
#4It's not clear in the code how you're batching the writes. Could the performance improvement be mostly explained by batches?
[deleted]
Re: SQLite improving performance with pre-sort
#5Depending on the size of the blob you might not want to use `WITHOUT ROWID`. I had a similar table with uuid keys and blobs of 40kb size on average and `WITHOUT ROWID` really tanked performance.
Re: SQLite improving performance with pre-sort
#6It's not clear in the code how you're batching the writes. Could the performance improvement be mostly explained by batches?
Looks like he’s doing 1M INSERTs per transaction.
In his 2nd example he’s doing the same thing, except he first sorts the data for each of those 1M INSERTs, leading to a 2-3x speedup.
Re: SQLite improving performance with pre-sort
#7[flagged]
Re: SQLite improving performance with pre-sort
#8This is not unique to SQLite... it should help with any database that uses B+Trees. Batch deletes are also typically faster when the IDs to be deleted are sorted in these same systems.
Re: SQLite improving performance with pre-sort
#9[flagged]