171–180 of 200 posts
Earlier quoted context omitted.
> (or indeed browser to browser would be possible both manually or over WebRTC) I have just done something similar in the past week, but without the WAL. It's pretty much an alternative to online spreadsheets for me. http://github.com/adhamsalama/sqlite-wasm-webrtc
This is a very interesting demo.
WebSQL should've just been Sqlite and the whole offline-first (and general app storage) ecosystem would've been so much nicer. Is there any hope of that happening? Instead of abstracting and over specifying sqlite, can the spec just specify a version of the SQLite API browsers should support and roll the version periodically?
"Rolling the version periodically" is probably quite problematic for browsers. Kind of a key point of the web is that stuff if at all possible keeps working. Breaking changes like that are hard. Even if the spec just listed occasional version and the webpage could choose which one; that means a potentially tricky maintenance burden on browser to support old versions of a potentially no longer supported sqlite, and ea…
https://news.ycombinator.com/item?id=15670808
I thought the reasons given for not moving forward with standardizing WebSQL and using a SQLite implementation were (and undoubtedly still are) very, very stupid so I'm not the right person to represent them here.
WebSQL should've just been Sqlite and the whole offline-first (and general app storage) ecosystem would've been so much nicer. Is there any hope of that happening? Instead of abstracting and over specifying sqlite, can the spec just specify a version of the SQLite API browsers should support and roll the version periodically?
I for one am glad WebSQL didn't establish itself. Now we get the most recent version of SQLite when we need it as a 410KB compressd WASM blob, as opposed to being stuck on browser-mandated versions of SQLite that might even be a decade old at this point.
Earlier quoted context omitted.
Mozilla refused to support it because then every implementation would have simply used SQLite, which would have promoted any implementation details to a de facto standard. (Even caniuse erroneously describes the feature as "allows SQLite database queries".) From the latest spec [1]: > The specification reached an impasse: all interested implementors have used the same SQL backend (Sqlite), but we need multiple indepe…
Did they really assume that they were going to be able to _restandardize_ SQL? No wonder IndexDB is hot useless garbage. The standardization issues around SQL already exist, are already widely known, and where common workarounds are already in practice. It's also an open source project that could have _easily_ incorporated compatibility code for this specific use case anyways. They made blind fealty to process more i…
As I pointed out in a separate comment:
Richard Hipp committed to creating and maintaining a flag in SQLite that would force SQLite to use whatever subset of SQL the WebSQL people settled on for their API. That would have of course worked independently of the SQLite version. (He also offered to write the SQL part of the spec.)
https://news.ycombinator.com/item?id=41860067
It was such a lost opportunity. I'm sure the SQLite guys would still be happy to do the work, but there certainly isn't any momentum on the browser side anymore.
The CORS restrictions / needing SharedArrayBuffer support kinda stinks. There is no way to use Sqlite3 off-thread without memory sharing? Couldn't postMessage work to pass data to the sqlite thread by using the third Transfer argument? Would postMessage transfer allow memory to be stored in a sqlite wasm database running a worker off-thread? Refering to this implementation's docs: https://github.com/sqlite/sqlite-was…
> The CORS restrictions / needing SharedArrayBuffer support kinda stinks. We have no CORS restrictions but one specific (and optional) VFS requires COOP/COEP for SharedArrayBuffer. If SharedArrayBuffer isn't available, that VFS won't load, but the rest of the library will plod along just fine: https://sqlite.org/wasm/doc/tip/persistence.md
Something that would be really fun would be to run SQLite in-memory in a browser but use the same tricks as Litestream and Cloudflare Durable Objects ( https://simonwillison.net/2024/Oct/13/zero-latency-sqlite-st... ) to stream a copy of the WAL log to a server (maybe over a WebSocket, though intermittent fetch() POST would work too). Then on subsequent visits use that server-side data to rehydrate the client-side da…
There are many layers of that's not how it works at play here. In-memory SQLite databases don't use WAL. Wasm (and browser Wasm, in particular) doesn't support anything like the shared memory APIs SQLite wants for its WAL mode. Litestream requires a very precise WAL setup to work (which just so happens to work with the default native SQLite setup, but is hard to replicate with Wasm). Cloudflare Durable Objects may ha…
Seems easy to me, just store/stream all update sql statements. That should have all information needed and don't have to mess with WAL data format etc.
Earlier quoted context omitted.
> do not define the use cases I genuinely don't mean to sound rude, and maybe I misunderstand, but how do you build software if you're not doing it with use cases in mind?
Usage is laid out well in the docs, I‘m not sure GP has read them. IMO it’s obvious that many use cases have been kept in mind. Defining a use case != accommodating a use case
Earlier quoted context omitted.
Then you don't need WAL. I didn't say replication isn't possible, I said "that's not how it works." A bunch of different people solved this already (though none in a couple of weekends); common to all of them: they don't use WAL mode.
> The shadow WAL is a directory next to your SQLite database where WAL files are effectively recreated as a sequence. . . These WAL files contain the original WAL frames & checksums to ensure consistency. https://litestream.io/how-it-works/ > (though none in a couple of weekends) As I said, getting sqlite in wasm to run in WAL mode such that we could start to implement replication, like for example how litestream doe…
Litestream works out-of-process. It needs shared memory WAL and file locking to work exactly like on "desktop" SQLite for it to even function and produce this "shadow WAL." No ifs, no buts, no inbetweens.
You just don't build Litestream without shared memory WAL. That's my entire point. These pieces you seem to think can be made to simply work together can't, in fact, work together at all, much less simply.
Exclusive locking mode WAL doesn't work with Litestream, and can't be made to work with Litestream, even with a little elbow grease.
Earlier quoted context omitted.
There are many layers of that's not how it works at play here. In-memory SQLite databases don't use WAL. Wasm (and browser Wasm, in particular) doesn't support anything like the shared memory APIs SQLite wants for its WAL mode. Litestream requires a very precise WAL setup to work (which just so happens to work with the default native SQLite setup, but is hard to replicate with Wasm). Cloudflare Durable Objects may ha…
"The general idea of streaming changes from SQLite would work, but it's a lot of work" Seems easy to me, just store/stream all update sql statements. That should have all information needed and don't have to mess with WAL data format etc.