[dead]
I benchmarked six Go SQLite drivers
21–30 of 64 posts
Re: I benchmarked six Go SQLite drivers
#22In my personal opinion and usage, the performance doesn't matter. Only one driver is written in pure go, and can be easily statically compiled and/or cross-compiled. > modernc, modernc.org/sqlite, a pure Go solution. This is a newer library, based on the SQLite C code re-written in Go. Unless I'm mistaken, this is not a re-write in Go. This is a transpilation of the the SQLite C library into go, using https://gitlab.…
The biggest issue of the modernc approach is that it can't use the proprietary sqlite test suites and fuzzer to ensure that it works like the original: https://www.sqlite.org/testing.html
(I just realized that probably goes for libsql as well)
Re: I benchmarked six Go SQLite drivers
#23the thing about the mattn driver is that it supports all the features that sqlite itself supports. you can compile in vtables, extra stat stuff, which FTS option you want, anything. and if you use zig as your cross compiler you don't even need separate toolchains for the different arch/OS combos, it all just works
Re: I benchmarked six Go SQLite drivers
#24I hadn't heard of sqinn before. According to these benchmarks it beats Cgo-based solutions most of the time, which makes it a very interesting candidate. Anyone have real-world experience using it?
Sqinn author here. Yes, sqinn performs quite well compared to the 'standard' mattn driver. The only use case it clearly breaks down is when SELECTing very large (gigabytes) resultsets.
That stood out to me as well. Any insights why sqinn and zombie underperform in this case, and is the problem inherent to their design?
Re: I benchmarked six Go SQLite drivers
#25Sure showing time and N works. But it'd be a lot easier to interpret the data quickly if it was shown in terms of operations per second. Smaller bars do not usually mean improved performance.
… what? Lots of graphs for performance (especially latency) use smaller bars to mean better numbers.
Re: I benchmarked six Go SQLite drivers
#26Earlier quoted context omitted.
Sqinn author here. Yes, sqinn performs quite well compared to the 'standard' mattn driver. The only use case it clearly breaks down is when SELECTing very large (gigabytes) resultsets.
That stood out to me as well. Any insights why sqinn and zombie underperform in this case, and is the problem inherent to their design?
For sqinn it's because of its design: Shuffling that much data over process boundaries takes time. For zombie, more pprof would be needed to explain the behaviour.
Re: I benchmarked six Go SQLite drivers
#27In my personal opinion and usage, the performance doesn't matter. Only one driver is written in pure go, and can be easily statically compiled and/or cross-compiled. > modernc, modernc.org/sqlite, a pure Go solution. This is a newer library, based on the SQLite C code re-written in Go. Unless I'm mistaken, this is not a re-write in Go. This is a transpilation of the the SQLite C library into go, using https://gitlab.…
My only complain so far about the package is that it required transpiled versions of the whole dependency chain for sqlite in order to get it working, from libc to tcl. The whole tree for the latest version is about 2G. This is not a trivial amount of traffic/space to use for every compilation.
Re: I benchmarked six Go SQLite drivers
#28In my personal opinion and usage, the performance doesn't matter. Only one driver is written in pure go, and can be easily statically compiled and/or cross-compiled. > modernc, modernc.org/sqlite, a pure Go solution. This is a newer library, based on the SQLite C code re-written in Go. Unless I'm mistaken, this is not a re-write in Go. This is a transpilation of the the SQLite C library into go, using https://gitlab.…
> Only one driver is written in pure go, and can be easily statically compiled and/or cross-compiled.
Which one do you mean? The WASM one? It includes WASM, it needs to be compiled too.
Ah github.com/cvilsmeier/sqinn-go. Which... is made by the same person that made this benchmark...?
edit: and that requires some random binary to be pre-installed...? Which is in C anyway?
https://github.com/cvilsmeier/sqinn-go
https://github.com/cvilsmeier/sqinn
so I don't see any "actually written in go".
Re: I benchmarked six Go SQLite drivers
#29In my personal opinion and usage, the performance doesn't matter. Only one driver is written in pure go, and can be easily statically compiled and/or cross-compiled. > modernc, modernc.org/sqlite, a pure Go solution. This is a newer library, based on the SQLite C code re-written in Go. Unless I'm mistaken, this is not a re-write in Go. This is a transpilation of the the SQLite C library into go, using https://gitlab.…
I cross compile mattn/go-sqlite3 a lot. It's about the easiest cgo project to cross compile since it has no external dependencies. Just use zig as your c compiler and it all just works.
Do you have any links for using Zig for this?
Re: I benchmarked six Go SQLite drivers
#30In my personal opinion and usage, the performance doesn't matter. Only one driver is written in pure go, and can be easily statically compiled and/or cross-compiled. > modernc, modernc.org/sqlite, a pure Go solution. This is a newer library, based on the SQLite C code re-written in Go. Unless I'm mistaken, this is not a re-write in Go. This is a transpilation of the the SQLite C library into go, using https://gitlab.…
Sqlite developers go to very great lengths making sure their software works reliably. You can't just introduce some transpiler in-between and expect that it'll work. It's a different software and it should not even be called sqlite IMO. The only proper way to use sqlite is to use FFI.
They run the whole testsuite on the transpiled code. I think.
But yeah it does give me a little shiver to think you are transpiling C code to go code, and the go-code is not really "pure go", but has a platform-dependent unsafe operations. Just look at the repo
https://gitlab.com/cznic/sqlite/-/tree/master/lib?ref_type=h...