181–190 of 200 posts
WebSQL should've just been Sqlite and the whole offline-first (and general app storage) ecosystem would've been so much nicer. Is there any hope of that happening? Instead of abstracting and over specifying sqlite, can the spec just specify a version of the SQLite API browsers should support and roll the version periodically?
Earlier quoted context omitted.
> The CORS restrictions / needing SharedArrayBuffer support kinda stinks. We have no CORS restrictions but one specific (and optional) VFS requires COOP/COEP for SharedArrayBuffer. If SharedArrayBuffer isn't available, that VFS won't load, but the rest of the library will plod along just fine: https://sqlite.org/wasm/doc/tip/persistence.md
Persistence is the whole point to me for a Database. OPFS finally adding a filesystem to the browser made seem like the web file database situation could finally standardize but COOP/COEP ruins it.
Earlier quoted context omitted.
> So SQLite in Wasm is just doing compute, and I do all the OS level stuff in Go. No need for Wasm concurrency, cause I can load multiple instances of my Wasm which act like independent OS processes that communicate through the filesystem (SQLite excels at this). Interesting. So when I am running concurrent readers using your package, it is just loading multiple instances of the wasm code? (I bottleneck to a single w…
Yes. Each connection lives in its own isolated sandbox, and only communicates with other connections through the “file system” (which is a virtual abstraction, actually). WAL mode is the “exception”: a few pages of the sandbox's memory are mapped to a file, and shared by all connections to the same database. Each sandbox is single threaded, and mostly lock free, does all its business in the calling goroutine, and reg…
Earlier quoted context omitted.
There are many layers of that's not how it works at play here. In-memory SQLite databases don't use WAL. Wasm (and browser Wasm, in particular) doesn't support anything like the shared memory APIs SQLite wants for its WAL mode. Litestream requires a very precise WAL setup to work (which just so happens to work with the default native SQLite setup, but is hard to replicate with Wasm). Cloudflare Durable Objects may ha…
"The general idea of streaming changes from SQLite would work, but it's a lot of work" Seems easy to me, just store/stream all update sql statements. That should have all information needed and don't have to mess with WAL data format etc.
I've been really interested in the local-first landscape lately but embedding SQLite seems really heavy-weight compared to using the browser's built-in storage APIs (in particular, IndexedDB) and it seems to be what most of the main open source libraries do. I'm interested to see a open-source solution (with sync) which provides an SQLite-like API but for the browser's native storage rather than trying to embed anoth…
Disclaimer: I'm the author. But you might be interested in TinyBase.
What are your thoughts about search over the store? Hydrate a separate lunr index?
And finally has anybody tried using your syncing over webrtc?
Earlier quoted context omitted.
> The shadow WAL is a directory next to your SQLite database where WAL files are effectively recreated as a sequence. . . These WAL files contain the original WAL frames & checksums to ensure consistency. https://litestream.io/how-it-works/ > (though none in a couple of weekends) As I said, getting sqlite in wasm to run in WAL mode such that we could start to implement replication, like for example how litestream doe…
Seriously? Litestream works out-of-process. It needs shared memory WAL and file locking to work exactly like on "desktop" SQLite for it to even function and produce this "shadow WAL." No ifs, no buts, no inbetweens. You just don't build Litestream without shared memory WAL. That's my entire point. These pieces you seem to think can be made to simply work together can't, in fact, work together at all, much less simply…
Seriously bro, take a massive chill pill. You have been way overcommitted to the "it's all impossible!!! How stupid to imagine! If only you were smart like me you'd see everything is impossible!" bit way too hard. Take a breather from your possessed absolute self certainty & derogatory belittling of others.
There's also lots of different fs impls for wasm! Just because some limitations apply to emscriptens impl doesn't mean another impl has the same limitations, or that we can't improve or work around. Use some of your hard working loves-to-tax-itself big brain to find avenues of possibility, rather than just pissing on every possibility.
Earlier quoted context omitted.
Seriously? Litestream works out-of-process. It needs shared memory WAL and file locking to work exactly like on "desktop" SQLite for it to even function and produce this "shadow WAL." No ifs, no buts, no inbetweens. You just don't build Litestream without shared memory WAL. That's my entire point. These pieces you seem to think can be made to simply work together can't, in fact, work together at all, much less simply…
Litestreams how-we-did-this to says they take a lock on the whole db during checkpoints to work, so I don't see why shared would be needed. Seriously bro, take a massive chill pill. You have been way overcommitted to the "it's all impossible!!! How stupid to imagine! If only you were smart like me you'd see everything is impossible!" bit way too hard. Take a breather from your possessed absolute self certainty & dero…
But that'll require more than looking at docs reading the words "lock" and "shared" and assuming you know what those mean.
Earlier quoted context omitted.
> The CORS restrictions / needing SharedArrayBuffer support kinda stinks. We have no CORS restrictions but one specific (and optional) VFS requires COOP/COEP for SharedArrayBuffer. If SharedArrayBuffer isn't available, that VFS won't load, but the rest of the library will plod along just fine: https://sqlite.org/wasm/doc/tip/persistence.md
Persistence is the whole point to me for a Database. OPFS finally adding a filesystem to the browser made seem like the web file database situation could finally standardize but COOP/COEP ruins it.
We have two OPFS-based VFSes. One requires COOP/COEP and one does not. Each makes feature trade-offs, though, so they're not equivalent.
We also offer persistence via localStorage and sessionStorage, so OPFS is not (for small databases) required for persistence.