Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sqlite3 WebAssembly

sqlite.org

Post reply on HN
Print view View original

121–130 of 200 posts

Re: Sqlite3 WebAssembly

#121

Post by sgbeal »

akira2501 wrote: post #81
debugnik wrote: post #58

Earlier quoted context omitted.

Mozilla refused to support it because then every implementation would have simply used SQLite, which would have promoted any implementation details to a de facto standard. (Even caniuse erroneously describes the feature as "allows SQLite database queries".) From the latest spec [1]: > The specification reached an impasse: all interested implementors have used the same SQL backend (Sqlite), but we need multiple indepe…

Did they really assume that they were going to be able to _restandardize_ SQL? No wonder IndexDB is hot useless garbage. The standardization issues around SQL already exist, are already widely known, and where common workarounds are already in practice. It's also an open source project that could have _easily_ incorporated compatibility code for this specific use case anyways. They made blind fealty to process more i…

> Did they really assume that they were going to be able to _restandardize_ SQL?

The core issue was not the SQL, but the underlying storage engine.

Microsoft's office doc formats aside, standards bodies, as a general rule, require that a standard be built around multiple independent, conformant implementations. WebSQL did not measure up there because all vendors implementing it chose the only viable option they had for the underlying storage: SQLite.

Re: Sqlite3 WebAssembly

#122

Post by infogulch »

digdugdirk wrote: post #98
wyager wrote: post #86

Earlier quoted context omitted.

WAL: Write ahead log, common strategy for DBs (sqlite, postgres, etc.) to improve commit performance. Instead of fsync()ing every change, you just fsync() a log file that contains all the changes and then you can fsync() the actual changes at your leisure Shared memory API: If you want to share (mutable) data between multiple processes, you need some kind of procedure in place to manage that. How do you get a referen…

Thank you! And side note on your last point - I've been burned too many times by confident hallucinations to trust my foundational learning to GPT. I hope someday that will improve, but for now ChatGPT is as trustworthy as an evening chat with someone at the bar. ... Someone who has been drinking since happy hour.

I was curious and Grok 2 seemed to do pretty good: https://x.com/i/grok/share/c2qCdF2wwIx7AHz0U1f2u8dTO

Re: Sqlite3 WebAssembly

#123

Post by infogulch »

T-Winsnes wrote: post #88
PUSH_AX wrote: post #76

Earlier quoted context omitted.

What’s the catch with this thing?

The security model is challenging, as it relies on Postgres users for iam. Your users essentially log directly into your db

You can see what this means specifically from the docs: https://electric-sql.com/docs/guides/auth

Re: Sqlite3 WebAssembly

#124

Post by akira2501 »

sgbeal wrote: post #121
akira2501 wrote: post #81

Earlier quoted context omitted.

Did they really assume that they were going to be able to _restandardize_ SQL? No wonder IndexDB is hot useless garbage. The standardization issues around SQL already exist, are already widely known, and where common workarounds are already in practice. It's also an open source project that could have _easily_ incorporated compatibility code for this specific use case anyways. They made blind fealty to process more i…

> Did they really assume that they were going to be able to _restandardize_ SQL? The core issue was not the SQL, but the underlying storage engine. Microsoft's office doc formats aside, standards bodies, as a general rule, require that a standard be built around multiple independent, conformant implementations. WebSQL did not measure up there because all vendors implementing it chose the only viable option they had f…

I've seen and understood that in hardware. I'm less convinced it's reasonable in this context. If there wasn't an _independent_ implementation it might barely make sense, but given that sqlite exists separately and can be installed separately, this is more of a "package management" or "system library access" problem than it is one of actual standardization by an independent body.

Which if you standardized this generic interface you could have multiple independent database engines available not just ones that are SQL based, although, SQL in particular was readily available through SQLite, and would clearly be exceptionally popular.

Huge miss when organizations stick the the routine rather then take an opportunity to explore and examine new ways forward. Meanwhile everyone is being held hostage by V2 to V3 manifest changes by _one_ vendor.

Re: Sqlite3 WebAssembly

#125

Post by infogulch »

ncruces wrote: post #73
koeng wrote: post #26

For use in Golang, I really like ncruces wasm SQLite package - https://github.com/ncruces/go-sqlite3 . Unlike cznic's go package (which is great, btw), the wasm version works well on OpenBSD and the like.

Author here. If you're interested, do ask questions.

Hey how's the mmap-based shared memory WAL approach working out? It's been about half a year since you finished the implementation and I only see one issue about it on the tracker, a good sign?

Re: Sqlite3 WebAssembly

#126

Post by arunaugustine »

jamesgpearce wrote: post #93
chrysoprace wrote: post #84

I've been really interested in the local-first landscape lately but embedding SQLite seems really heavy-weight compared to using the browser's built-in storage APIs (in particular, IndexedDB) and it seems to be what most of the main open source libraries do. I'm interested to see a open-source solution (with sync) which provides an SQLite-like API but for the browser's native storage rather than trying to embed anoth…

Disclaimer: I'm the author. But you might be interested in TinyBase.

TinyBase looks very promising! Is there a doc or reference you can guide me to, for using TinyBase with Preact.js instead of React?

Re: Sqlite3 WebAssembly

#127

Post by throwaway2037 »

digdugdirk wrote: post #82
ncruces wrote: post #68

Earlier quoted context omitted.

There are many layers of that's not how it works at play here. In-memory SQLite databases don't use WAL. Wasm (and browser Wasm, in particular) doesn't support anything like the shared memory APIs SQLite wants for its WAL mode. Litestream requires a very precise WAL setup to work (which just so happens to work with the default native SQLite setup, but is hard to replicate with Wasm). Cloudflare Durable Objects may ha…

As someone who uses sqlite fairly regularly, but doesn't understand what most of those paragraphs mean, do you have any recommendations for learning resources? I'm gathering that I need to learn about: - WAL - Shared Memory APIs - Concurrency models - Durable Objects?

WAL - Shared Memory APIs: https://sqlite.org/wal.html

concurrency model of in-browser Wasm: https://medium.com/@quincarter/wasm-is-awsm-and-multi-thread...

Cloudflare Durable Objects: https://www.cloudflare.com/developer-platform/durable-object...

Re: Sqlite3 WebAssembly

#128

Post by outlore »

matlin wrote: post #114
outlore wrote: post #5

i’ve been looking for a Tanstack Query style library that is backed by Sqlite (backed by OPFS or some other browser storage) and syncs with an API in the background. Does anything like that exist? i’ve seen ElectricSQL and other sync engines but they are a bit opinionated. I’m pretty new to local-first but i feel like the developer ergonomics are not quite there yet Meanwhile for “local-only” it would be great to use…

I've made just this! Docs for it: https://www.triplit.dev/docs/frameworks/tanstack-router#exam... It by default uses IndexedDB but can also use SQLite but does real time, relational querying and (optionally) syncs with you server. Re: developer ergonomics, this is our primary focus so I don't love to get your feedback on!

thanks for sharing! will take a look :)

Re: Sqlite3 WebAssembly

#129

Post by outlore »

gagik_co wrote: post #21
outlore wrote: post #5

i’ve been looking for a Tanstack Query style library that is backed by Sqlite (backed by OPFS or some other browser storage) and syncs with an API in the background. Does anything like that exist? i’ve seen ElectricSQL and other sync engines but they are a bit opinionated. I’m pretty new to local-first but i feel like the developer ergonomics are not quite there yet Meanwhile for “local-only” it would be great to use…

I have working to replicate TanStack query experience by writing my own queries wrapped around PowerSync, although actually on Flutter (using Flutter Hooks! which was cool to use coming from React). It’s a very internal design meant for my app tetr[1] right now (and actually being migrated over from Realm). I am hoping to potentially standardize it and publish a package once it’s mature enough but not too wrap to mak…

very cool product page!

Re: Sqlite3 WebAssembly

#130

Post by hiccuphippo »

simonw wrote: post #32

Something that would be really fun would be to run SQLite in-memory in a browser but use the same tricks as Litestream and Cloudflare Durable Objects ( https://simonwillison.net/2024/Oct/13/zero-latency-sqlite-st... ) to stream a copy of the WAL log to a server (maybe over a WebSocket, though intermittent fetch() POST would work too). Then on subsequent visits use that server-side data to rehydrate the client-side da…

This sounds like what meteor was 10+ years ago but with sqlite instead of mongodb.
Post reply on HN