This is great to see, and a project I considered attempting myself for a bit. I’m excited to test it out.
@jlongster I have a question about this:
> The backend calls it [Atomics.wait] to wait on the result from the worker and blocks until it’s done.
Does this mean the main (UI) thread is blocked during queries? Or are there more threads, like UI SQLite main SQLite FS backend?
————
At Notion, we’ve used IndexedDB for two purposes: (1) to durably persist a queue of changes to send to our backend, and (2) in the desktop app, to LRU cache the page data we read from the server to accelerate reads. Both of these used localStorage years ago, but we ported to IndexedDB because of data loss on localStorage. Porting was fine for the write queue, but we really noticed the slow when we tried porting the data cache. To get close to the original performance we coalesce reads, and we delay writes to the cache significantly so they can batch more effectively into a single readwrite transaction that we send after the reads for the current page load are complete.
That worked okay, but it was annoying to maintain the IDB cache code because our Android and iOS apps used SQLite for their caches, and it’s so much easier to add new queries using SQL compared to writing IDB iterations - and it’s faster. So we switched to using native SQLite via a bridge to a Node process. Now with absurd-sql, maybe we could bring the same caching logic to browsers.
The thing stopping me is how unreliable we’ve found IndexedDB to be - aside from the optimization work. We notice a lot of bugs in IDB implementations on different browsers. In Safari (especially on iOS) there’s a bunch of spooky issues that have caused stalls or spurious errors, sometimes requiring an app restart before the IDB database can be re-opened. Forget it on Android - weird vendor webview patches mean your storage might get cleared out from under you. On Firefox, we notice that sometimes the IndexedDB database doesn’t create all the object stores we request for some reason. Even on Chrome, IndexedDB can suddenly start refusing writes in the middle of a session with no clear explanation, and on Windows restarting the computer is sometimes the only fix.
If we can share SQLite queries with our native apps then maybe it’s worth wading deeper into these issues… but it really does feel like building on quicksand.