21–30 of 66 posts
If you really want to use SQLite without anything that isn't Go (since this project involves forking and communicating with a non-Go SQLite in a separate process over pipes), there's a Go translation of SQLite's C source. :) https://gitlab.com/cznic/sqlite https://datastation.multiprocess.io/blog/2022-05-12-sqlite-i...
Earlier quoted context omitted.
What’s the case against cgo for SQLite? Just the usual cgo performance overhead? It seems like a pretty good cgo use case: a decent amount of work, which is typically slow enough that cgo overhead isn’t perf critical (because DB usually means disk reads), a super robust and well tested C library with a super well maintained cgo wrapper (mattn).
mattn's sqlite3 is probably the most ideal use case for cgo imaginable. But it can still be annoying to set up cgo to build across many platforms. I think Go should just pull a Zig and just embed a full blown C compiler into go build.
Using this cznic's pure-Go sqlite saved the day.
If you really want to use SQLite without anything that isn't Go (since this project involves forking and communicating with a non-Go SQLite in a separate process over pipes), there's a Go translation of SQLite's C source. :) https://gitlab.com/cznic/sqlite https://datastation.multiprocess.io/blog/2022-05-12-sqlite-i...
Earlier quoted context omitted.
To be a bit more specific here: pure Go binaries are trivial to cross-compile and they Just Work™ basically all the time. `GOOS="darwin" GOARCH="arm64" go build .` and you're done. Just iterate over the combinations you care about, they'll all work. As soon as you or a library touches cgo, you have to deal with finding and setting up C cross-compilation tooling for your target(s), dynamic linking details, tons of stu…
Golang works on Plan9. It can even bootstrap itself. A few months ago I was trying to setup some server software on 9Front for giggles and while most stuff worked I couldn't past the Sqlite CGO dependencies.
You'll need to use the sqlite3_nolock build tag; concurrent writes will quickly corrupt your database. SetMaxOpenConns(1) is your friend.
But it should work. I'm interested if it doesn't. Feedback appreciated.
If you really want to use SQLite without anything that isn't Go (since this project involves forking and communicating with a non-Go SQLite in a separate process over pipes), there's a Go translation of SQLite's C source. :) https://gitlab.com/cznic/sqlite https://datastation.multiprocess.io/blog/2022-05-12-sqlite-i...
The logic of easy cross compiles doesn't really hold up for go translated SQLite. It depends on a huge pile of per platform support code, of varying quality. If you're only going to target known working platforms, may as well use cgo and a known working cross compiler.
That's the elevator pitch (so far) for: https://github.com/ncruces/go-sqlite3/tree/main
Earlier quoted context omitted.
The logic of easy cross compiles doesn't really hold up for go translated SQLite. It depends on a huge pile of per platform support code, of varying quality. If you're only going to target known working platforms, may as well use cgo and a known working cross compiler.
Does compiling to WASM, using a cgo free WASM runtime, and replacing the OS layer (VFS) with portable Go code count? That's the elevator pitch (so far) for: https://github.com/ncruces/go-sqlite3/tree/main
Modernc/libc takes a rather different approach. It's got some pretty funny files in it. https://gitlab.com/cznic/libc/-/blob/master/musl_openbsd_arm...
Earlier quoted context omitted.
Does compiling to WASM, using a cgo free WASM runtime, and replacing the OS layer (VFS) with portable Go code count? That's the elevator pitch (so far) for: https://github.com/ncruces/go-sqlite3/tree/main
Yeah, that's cool. :) although probably a bit slower running through an interpreter. Modernc/libc takes a rather different approach. It's got some pretty funny files in it. https://gitlab.com/cznic/libc/-/blob/master/musl_openbsd_arm...
The WASM runtime https://wazero.io has a compiler on amd64 and arm64 (on Linux, macOS, Windows, and FreeBSD), but the current compiler while very fast (at compiling), is very naive (generates less than optimal code).
An optimizing compiler is currently being developed, and should be released in the coming months. I'm optimistic that this compiler will cover the performance gap between WASM and modernc.
Earlier quoted context omitted.
What’s the case against cgo for SQLite? Just the usual cgo performance overhead? It seems like a pretty good cgo use case: a decent amount of work, which is typically slow enough that cgo overhead isn’t perf critical (because DB usually means disk reads), a super robust and well tested C library with a super well maintained cgo wrapper (mattn).
mattn's sqlite3 is probably the most ideal use case for cgo imaginable. But it can still be annoying to set up cgo to build across many platforms. I think Go should just pull a Zig and just embed a full blown C compiler into go build.
If you really want to use SQLite without anything that isn't Go (since this project involves forking and communicating with a non-Go SQLite in a separate process over pipes), there's a Go translation of SQLite's C source. :) https://gitlab.com/cznic/sqlite https://datastation.multiprocess.io/blog/2022-05-12-sqlite-i...
It’s not 100% Go, though. It forks a process that manages the sqlite file. Since the communication with it is written in go, it avoids using cgo.