Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Sqinn-Go is a Golang library for accessing SQLite databases in pure Go

github.com

Post reply on HN
Print view View original

51–60 of 66 posts

Re: Show HN: Sqinn-Go is a Golang library for accessing SQLite databases in pure Go

#51

Post by nighmi »

cvilsmeier wrote: post #47
twodave wrote: post #36

Nice work! It's fun to see other people hacking away at Golang projects. I've been contemplating how to write tests for my own project, and this has given me some ideas.

Thank you! Nice to hear :-) Use the code any way you want, it's unlicensed.

Unlicensed is the same as fully copyrighted. There is a presumotion of ownership. Licenses in this case serve to clarify allowable uses. Without a license, nothing is allowable as you maintain the right to do anything within a copyright holder's legal right.

https://www.synopsys.com/blogs/software-security/unlicensed-...

Re: Show HN: Sqinn-Go is a Golang library for accessing SQLite databases in pure Go

#52

Post by thde »

nighmi wrote: post #51
cvilsmeier wrote: post #47

Earlier quoted context omitted.

Thank you! Nice to hear :-) Use the code any way you want, it's unlicensed.

Unlicensed is the same as fully copyrighted. There is a presumotion of ownership. Licenses in this case serve to clarify allowable uses. Without a license, nothing is allowable as you maintain the right to do anything within a copyright holder's legal right. https://www.synopsys.com/blogs/software-security/unlicensed-...

The project uses The Unlicense:

https://github.com/cvilsmeier/sqinn-go/blob/master/LICENSE

Re: Show HN: Sqinn-Go is a Golang library for accessing SQLite databases in pure Go

#53

Post by sureglymop »

cvilsmeier wrote: post #50
Animats wrote: post #45

Earlier quoted context omitted.

If you're going to run the database in a background process, you may as well run MySQL or one of its derivatives, or Postgres. More concurrency.

> you may as well run MySQL You're right, there are use-cases where SQLite is not appropriate. But nothing beats the ease of installation/backup/maintenance of SQLite compared to server databases like MySQL or Postgres. Another point is development: For unit-tests, I found that initializing a SQLite database for each test-run is much easier (and faster) than having a VM that runs Postgres/MySQL/etc, which I have to s…

Eh, the backup is a bit of a pain... The sqlite3 tool supports a .backup command but it's basic and can't even write to stdout, only directly to a file. There is an online backup api though but most APIs and libraries built around sqlite don't even acknowledge it, let alone support it. Just copying the sqlite file is of course not a real backup and if done at the wrong time will lead to a corrupted backup.

Re: Show HN: Sqinn-Go is a Golang library for accessing SQLite databases in pure Go

#54

Post by cvilsmeier »

ComputerGuru wrote: post #10
parhamn wrote: post #8

Earlier quoted context omitted.

> Just the usual cgo performance overhead? No, the performance is certainly orders of magnitude faster than serializing over std streams on a subprocess (c ffi calls in cgo are 10s of nanoseconds). But one of the big draws of golang is the write-once-compile-anywhere toolchain and calls cgo makes that harder.

But compiling the non-go assistant process is not going to be any easier than cgo, right?

> But compiling the non-go assistant process is not going to be any easier than cgo, right?

Right. But you have to do it only once, or you can download a pre-built library from https://github.com/cvilsmeier/sqinn/releases (windows/amd64 or linux/amd64)

Re: Show HN: Sqinn-Go is a Golang library for accessing SQLite databases in pure Go

#55

Post by guappa »

cvilsmeier wrote: post #48
guappa wrote: post #40

Earlier quoted context omitted.

Wow, that sounds incredibly inefficient!

> Wow, that sounds incredibly inefficient! That's exactly what I thought, too. But, to my own surprise, it's as fast as CGO solutions, in most cases even faster. You may check here: https://github.com/cvilsmeier/sqinn-go-bench

The benchmarks seem to put load on the db but not on the data.

I'd want a benchmark that is just select * from table, where table is a few gigabytes…

Re: Show HN: Sqinn-Go is a Golang library for accessing SQLite databases in pure Go

#56

Post by ncruces »

hnlmorg wrote: post #44
oefrha wrote: post #39

Earlier quoted context omitted.

Pulling a zig isn't going to solve all problems, as the zig cross compiler itself runs into problems fairly often. I already use CC='zig cc -target x86_64-linux-musl' (or whatever the target is) with cgo to cross-compile mattn/go-sqlite3, and relatively recently it simply stopped working[1] and a workaround had to do be added to about every single project of mine using SQLite through cgo. I also once tried to figure…

I’m going to add a counter argument to all the cgo views raised: Personally I’ve had much more portability problems with a lot of the native Go ports of sqlite than I have with mattn’s cgo library. I author a shell that targets most of the architectures and platforms supported by Go. At the request of some users, I added support for a Go native library because they didn’t want to install a C compilers as well as a Go…

Can you say more, like name your project?

I'm building https://github.com/ncruces/go-sqlite3 and am a community maintainer of https://wazero.io

I can cross compile SQLite into all platforms that Go OOB compiles too, with the caveat that any that aren't linux/windows/darwin/freebsd/illumos (CPU architecture doesn't matter) need a build flag because of file locking: https://github.com/ncruces/go-sqlite3/blob/main/vfs/README.m...

Anything that helps me test portability, or any feedback you might have on it, would be greatly appreciated.

Re: Show HN: Sqinn-Go is a Golang library for accessing SQLite databases in pure Go

#57

Post by ncruces »

hamandcheese wrote: post #18
eatonphil wrote: post #2

If you really want to use SQLite without anything that isn't Go (since this project involves forking and communicating with a non-Go SQLite in a separate process over pipes), there's a Go translation of SQLite's C source. :) https://gitlab.com/cznic/sqlite https://datastation.multiprocess.io/blog/2022-05-12-sqlite-i...

Would it be possible to link a pure go codebase to pre-compiled sqlite binaries and not need to worry about cgo when cross compiling?

The dynamic linking story in Go is complicated at best, particularly if you shun cgo, because you commonly need to cgo into ld.so to use dynamic linking.

But there's this if you wanna try: https://github.com/ebitengine/purego

Re: Show HN: Sqinn-Go is a Golang library for accessing SQLite databases in pure Go

#58

Post by scosman »

scosman wrote: post #13
infogulch wrote: post #6

Earlier quoted context omitted.

mattn's sqlite3 is probably the most ideal use case for cgo imaginable. But it can still be annoying to set up cgo to build across many platforms. I think Go should just pull a Zig and just embed a full blown C compiler into go build.

Exactly. Good to know I didn’t miss a trade off in the mix. If I’m choosing to use a C framework (SQLite) I’m okay signing up for the environment costs. Prefer that over abstractions in an intermediate layer that might not be maintained in a few years.

Just for the record, had to compile sqlite in gomobile literally the day after this comment and it was a big pain . But got it working and sticking with this approach.

Re: Show HN: Sqinn-Go is a Golang library for accessing SQLite databases in pure Go

#59

Post by tgv »

guappa wrote: post #55
cvilsmeier wrote: post #48

Earlier quoted context omitted.

> Wow, that sounds incredibly inefficient! That's exactly what I thought, too. But, to my own surprise, it's as fast as CGO solutions, in most cases even faster. You may check here: https://github.com/cvilsmeier/sqinn-go-bench

The benchmarks seem to put load on the db but not on the data. I'd want a benchmark that is just select * from table, where table is a few gigabytes…

As with all things software: you can't eat your cake and have it. This is a compromise which suits many use cases, but not all. I've experimented with the version that was translated from C to go, and it worked fine for me (low intensity service using the db for auth checks and logging). Sqinn-go will probably serve as well.

Re: Show HN: Sqinn-Go is a Golang library for accessing SQLite databases in pure Go

#60

Post by hnlmorg »

ncruces wrote: post #56
hnlmorg wrote: post #44

Earlier quoted context omitted.

I’m going to add a counter argument to all the cgo views raised: Personally I’ve had much more portability problems with a lot of the native Go ports of sqlite than I have with mattn’s cgo library. I author a shell that targets most of the architectures and platforms supported by Go. At the request of some users, I added support for a Go native library because they didn’t want to install a C compilers as well as a Go…

Can you say more, like name your project? I'm building https://github.com/ncruces/go-sqlite3 and am a community maintainer of https://wazero.io I can cross compile SQLite into all platforms that Go OOB compiles too, with the caveat that any that aren't linux/windows/darwin/freebsd/illumos (CPU architecture doesn't matter) need a build flag because of file locking: https://github.com/ncruces/go-sqlite3/blob/main/vfs/R…

Sorry, just seen this. It is https://murex.rocks

sqlite3 only needs to run from one thread in my use case. Which might explain why I have fewer problems than most.

Post reply on HN