Live data from Hacker News

Sqlite3 WebAssembly

sqlite.org

51–60 of 200 posts

Re: Sqlite3 WebAssembly

#51
post #47
post #41

Earlier quoted context omitted.

surprise! it's been there for decades: https://en.wikipedia.org/wiki/Web_SQL_Database

It was there for a decade: https://caniuse.com/sql-storage

I wonder why it was unmaintained/dropped. Was there something wrong with it, and if so, would that also apply to this kind of wasm implementation?

Re: Sqlite3 WebAssembly

#52
post #4

Slight point of confusion: that page says: > These components were initially released for public beta with version 3.40 and will tentatively be made API-stable with the 3.41 release, pending community feedback. But the most recent release of SQLite is 3.46.1 (from 2024-08-13) Presumably they are now "API-stable" but the page hasn't been updated yet. It would be great if the SQLite team published an official npm packa…

You can use DuckDB WASM independently of Pyodide and can extend it with SQLite.

Though it seems to be somewhat limited. I couldn't even check what version it has, since sqlite_version() was missing. Version in the repository [1] is 3.38.1, which is from quite a ways ago.

At the moment DuckDB web shell can't load SQLite extension, since that hasn't been released for yesterday's 1.1.2. Earlier version does work using recently updated WASM edition. That can be extended with spatial including GDAL, vector search etc [2]. Making your own "SQL web shell" wasn't too hard, though docs weren't quite complete enough for me.

[1] https://github.com/duckdb/sqlite_scanner/blob/main/src/sqlit... [2] https://github.com/duckdb/duckdb-wasm/releases/tag/v1.29.0

Re: Sqlite3 WebAssembly

#54

Earlier quoted context omitted.

1.3MB seems perfectly reasonable in a modern web app, especially since it will be cached after the first visit to the site. If you’re just storing user preferences, obviously don’t download SQLite for your web app just to do that… but if you’re doing something that benefits from a full database, don’t fret so much about 1MB that you go try to reinvent the wheel for no reason. If the other comment is correct, then it…

A megabyte here, a megabyte there, pretty soon you’re talking about a really heavyweight app.

We are past the stage where every piece of JS has to be loaded upfront and delay the first meaningful paint. Modern JS frameworks and module are chunked and can be eager/lazy loaded. Unless you make the sqlite DB integral part for your first meaningful page load, preloading those 1.3MB in the background/upon user request is easy.

Re: Sqlite3 WebAssembly

#55
WebSQL should've just been Sqlite and the whole offline-first (and general app storage) ecosystem would've been so much nicer.

Is there any hope of that happening? Instead of abstracting and over specifying sqlite, can the spec just specify a version of the SQLite API browsers should support and roll the version periodically?

Re: Sqlite3 WebAssembly

#57

I wasn't able to tell from a quick look through the page: could someone help me understand the use cases here? More specifically, would this be able to be a "replacement" for indexedDB? Does the data persist, or do I need to keep the sqlite file in the filesytemAPI (or indexedDB/localstorage) myself?

From the about page:

> Specific Goals of this Project

> Insofar as possible, support persistent client-side storage using available JS APIs. As of this writing, that includes the Origin-Private FileSystem (OPFS) and (very limited) storage via the window.localStorage and window.sessionStorage backend.

Re: Sqlite3 WebAssembly

#58
post #51
post #47

Earlier quoted context omitted.

It was there for a decade: https://caniuse.com/sql-storage

I wonder why it was unmaintained/dropped. Was there something wrong with it, and if so, would that also apply to this kind of wasm implementation?

Mozilla refused to support it because then every implementation would have simply used SQLite, which would have promoted any implementation details to a de facto standard. (Even caniuse erroneously describes the feature as "allows SQLite database queries".)

From the latest spec [1]:

> The specification reached an impasse: all interested implementors have used the same SQL backend (Sqlite), but we need multiple independent implementations to proceed along a standardisation path.

[1]: https://www.w3.org/TR/webdatabase/

This won't be a problem for wasm SQLite because it isn't a standard being shipped by browsers, just another dependency.

Re: Sqlite3 WebAssembly

#59
post #30

As a general question, in what scenarios is it more beneficial to send the full DB and let the browser handle the queries? Maybe phrased a better way - when would I use this to improve a user experience over the traditional server-hosted db model?

Personally I'm using it for a statically hosted website, so a server-hosted database was never an option. Also with the right driver, it's possible to stream the chunks of the database as needed rather than sending the full database: https://github.com/mmomtchev/sqlite-wasm-http

I can even do Sqlite's full text search without downloading the entire FTS database. Just most of it, if the search term is short enough.

Re: Sqlite3 WebAssembly

#60
post #19
post #3

So after downloading from the official downloads page and stripping away all the mjs files and "bundler-friendly" files, a minimal sqlite wasm dependency will be about 1.3MB. For an in-browser app, that seems a bit much but of course wasm runs in other places these days where it might make more sense.

It's pretty compressible at least, sqlite3.js+wasm are 1.3MB raw but minifying the JS and then compressing both files with Brotli gets them down to 410KB.

A lot of HTML's nowadays have 100 - 300 kb. That's only the HTML (!!).

Adding 400 for such a high quality piece of DB actually borders reasonability.

And makes me think: what the hell are frontend devs thinking!? Multiple MB's in JS for a news website. Hundreds of KB's for HTML. It's totally unreasonable.

Post reply on HN