71–80 of 200 posts
I used the wasm build of sqlite and the Chicory runtime to create a pure JVM executed sqlite library: https://github.com/dylibso/sqlite-zero It's more of an experiment than an attempt to make something production ready, though I could see it being useful to bring dependency-less sqlite tooling to the JVM ecosystem.
Earlier quoted context omitted.
> Presumably they are now "API-stable" but the page hasn't been updated yet. That's correct. i'll try my best to remember to update that reference the next time i'm back on the computer. > It would be great if the SQLite team published an official npm package Not a chance. We publish only vanilla JS and adamantly refuse to go down the rabit hole of supporting out-of-language tools (none of which any of our project me…
> We publish ONLY vanilla JS and adamantly refuse to go down rabit hole of supporting the frameworks du jour A bit confused at this, NPM is just a package manager / distribution mechanism, not a framework. Totally fair if you don't want to publish for all the package managers, though for Javascript there's only a few that are relevant. NPM has been around for a decade.
It's an out-of-language packaging/distribution framework (and it's not the only one). It's not part of the JS standards.
My comments above have been edited to reframe our stance on npm and frameworks in general.
For use in Golang, I really like ncruces wasm SQLite package - https://github.com/ncruces/go-sqlite3 . Unlike cznic's go package (which is great, btw), the wasm version works well on OpenBSD and the like.
How long until we see WebAssembly/WebGPU become a platform independent choice for deploying server side code as well?
SRE here, it's currently happening in a few parts but overall, it's not as attractive on server side. Server Side code running is mostly a solved problem and for very few organizations, the benefits of WASM don't outweigh any difficulties in getting it running.
I know what you mean here, but I think we're very limited in what we tend to run. Polyglot programming still isn't really a thing, and with things like WASI standardized (someday soon I hope), I could imagine it becoming a lot nicer.
I used the wasm build of sqlite and the Chicory runtime to create a pure JVM executed sqlite library: https://github.com/dylibso/sqlite-zero It's more of an experiment than an attempt to make something production ready, though I could see it being useful to bring dependency-less sqlite tooling to the JVM ecosystem.
What's the file system access like, WASI?
When I did this experiment a few months ago, what we could accomplish was pretty limited. I could load and query databases, but not write to them. However the Chicory wasip1 implementation is advancing.
BTW, we've borrowed a few ideas from wazero so thanks for your work there :)
Something that would be really fun would be to run SQLite in-memory in a browser but use the same tricks as Litestream and Cloudflare Durable Objects ( https://simonwillison.net/2024/Oct/13/zero-latency-sqlite-st... ) to stream a copy of the WAL log to a server (maybe over a WebSocket, though intermittent fetch() POST would work too). Then on subsequent visits use that server-side data to rehydrate the client-side da…
Are you thinking something like https://electric-sql.com/
For use in Golang, I really like ncruces wasm SQLite package - https://github.com/ncruces/go-sqlite3 . Unlike cznic's go package (which is great, btw), the wasm version works well on OpenBSD and the like.
Author here. If you're interested, do ask questions.
Context: Currently using go-duckdb and while it's working for us, getting rid of cgo would be a huge help. Would be quite interested myself to attempt this.
Earlier quoted context omitted.
What's the file system access like, WASI?
Chicory has some partial wasip1 support. https://github.com/dylibso/chicory/tree/main/wasi . We use jimfs to keep things simple and secure (and not worry about exposing the real filesystem): https://github.com/google/jimfs When I did this experiment a few months ago, what we could accomplish was pretty limited. I could load and query databases, but not write to them. However the Chicory wasip1 implementation is advan…
If the goal was pure Java SQLite¹, a VFS from scratch would be better.
I think since I started my Go/wazero effort, WASI+SQLite improved a bunch. I had to start with the demo VFS; the Unix VFS now builds. But custom VFS is still the way to go, IMO.
And thanks! My contributions to wazero were tiny. Best of luck with Chicory!
1: strong NestedVM vibes here; 11 years ago… gosh, I feel old now. https://stackoverflow.com/questions/18186507/pure-java-vs-na...
Earlier quoted context omitted.
Chicory has some partial wasip1 support. https://github.com/dylibso/chicory/tree/main/wasi . We use jimfs to keep things simple and secure (and not worry about exposing the real filesystem): https://github.com/google/jimfs When I did this experiment a few months ago, what we could accomplish was pretty limited. I could load and query databases, but not write to them. However the Chicory wasip1 implementation is advan…
If the goal is to improve Chicory WASI support, this is the way. If the goal was pure Java SQLite¹, a VFS from scratch would be better. I think since I started my Go/wazero effort, WASI+SQLite improved a bunch. I had to start with the demo VFS; the Unix VFS now builds. But custom VFS is still the way to go, IMO. And thanks! My contributions to wazero were tiny. Best of luck with Chicory! 1: strong NestedVM vibes here…
agreed, though this was more an experiment to test Chicory once we built initial wasi support. I'd love to see it picked up and improved. I think that's the direction I'd go if i want some kind of production ready library.
Earlier quoted context omitted.
Author here. If you're interested, do ask questions.
Very cool project! Do you know if this would be possible for duckdb? Is there something about sqlites APIs and wasm build that made it feasible? Context: Currently using go-duckdb and while it's working for us, getting rid of cgo would be a huge help. Would be quite interested myself to attempt this.
Wasm is fine for compute (though concurrency is still a somewhat open question).
To have Wasm talk to the outside world, you need “host calls” where the guest calls the host.
On a browser that's Wasm calling JavaScript. On my Go driver, it's Wasm calling Go.
For server side, there's also a standard set of “host calls” modeled around POSIX/Linux syscalls called WASI.
I could've build my project around WASI, but WASI is rather limited (and SQLite support for WASI was more limited even, it's improved a bit since). DuckDB might work out-of-the-box this way.
I, instead, took advantage of SQLite's architecture and replaced its VFS layer with one in Go: https://sqlite.org/vfs.html
So SQLite in Wasm is just doing compute, and I do all the OS level stuff in Go. No need for Wasm concurrency, cause I can load multiple instances of my Wasm which act like independent OS processes that communicate through the filesystem (SQLite excels at this).
As I said, I dunno how well all those decisions would map to DuckDB.