Live data from Hacker News

Go port of SQLite without CGo

gitlab.com

81–90 of 122 posts

Re: Go port of SQLite without CGo

#81
post #59
post #55

Earlier quoted context omitted.

Everyone _should_ also have 100% test coverage, canary deployments, and local dev parity. Unfortunately, the real world gets in the way and means we don’t all work with optimal setups.

In that case, it starts by having a local environment that matches the OS of the servers. Linux OEM vendors will appreciate it.

You’re running graviton arm locally? It’s all docker containers in the end, so that’s what you get.

Re: Go port of SQLite without CGo

#82

Beware: This library seems to have bugs that cause it to break on some writes. Recently held an event that heavily used this library and it broke on us half way through, had to wipe the database and switch it back to the cgo version (the data inside was mostly ephemeral, which was quite a relief).

I've noticed it has different transaction behavior than normal Sqlite, which can cause things to break more easily. I patch Go software that uses it to use the C library as a result, and I stopped having corruption problems.

Re: Go port of SQLite without CGo

#83
post #57

Earlier quoted context omitted.

Sure, but then DevOps aren't doing their jobs. No devs on production servers unless for tracking down issues, or servers configured for remote development. Which in both cases, don't require cross compilation to start with.

There are less and less touching servers directly happening. Not only DevOps but the current raise of k8s and serverless deployments making it less relevant. There is also the security aspect of it. The downside is that people (especially the ones who are just joining the workforce) are less efficient with the command line and more dependant on GUI. For example some of the junior devs do not know git cli anymore and…

Or worse, when what you think is a big actually is a feature. I basically use vscode git only for staging and merging and sometimes not even that, when it has issues with hunks on the beginning of the file or whatever.

Re: Go port of SQLite without CGo

#85
post #59

Earlier quoted context omitted.

In that case, it starts by having a local environment that matches the OS of the servers. Linux OEM vendors will appreciate it.

You’re running graviton arm locally? It’s all docker containers in the end, so that’s what you get.

No, and why should I?

I got into UNIX development back when the whole team shared a development server and we used telnet and X Windows for our "IDE".

No one was running SPARC, RISC System/6000, PA-RISC, MIPS, Eclipse MV locally.

When everyone keeps worshipping UNIX, maybe it is time to learn how grey beards used to develop for it.

Re: Go port of SQLite without CGo

#86
post #5

Earlier quoted context omitted.

It’s hard for Go to beat C in a straight benchmark - Go is garbage collected and allocations often have extra overhead due to the memory layout of interfaces.

Not only Go compiler is not as good at optimisation as C/C++/Rust/Zig/D compilers, but also Go FFI to C has significant overhead, which means there is often not much point in using low-level Linux APIs for performance advantage. Same problem like with Java. And in a database system you quite likely want a fine degree of control with non-standard stuff.

[deleted]

Re: Go port of SQLite without CGo

#87

It's a really neat project. I did some simple benchmarks of it and it was between 10% slower to twice as slow. Which depending on your use case isn't that bad! https://datastation.multiprocess.io/blog/2022-05-12-sqlite-i...

What makes it slower?

[dead]

Re: Go port of SQLite without CGo

#88
post #44
post #26

Earlier quoted context omitted.

In addition, a lot of optimizations (like loop unrolling) are left on the table to preserve Go's fast compile times.

Gccgo is the answer here.

Gccgo at its core may be fast but when it comes to how the go frontend interpret it to gcc ir that pales in comparison than just straight up optimizating the Go compiler

There’ll more opportunities for more optimizations since they move to SSA ir since 1.5 and in turn compile time will definitely go down

Re: Go port of SQLite without CGo

#89
post #72
post #8

If you cross-compile (which everybody using macOS to developer for Linux servers does), it's annoying to introduce SQLite to your project, because immediately the conventional `GOOS= GOARCH= go build` trick stops working. But, in case it's helpful, it's also really easy to set up a full cross-compiling environment, either with Zig or with Filippo's musl-cross: https://words.filippo.io/easy-windows-and-linux-cross-com…

The following post [0] describes how to build static binaries even with cgo enabled: GOOS= GOARCH= CGO_ENABLED=1 go build \ -tags osusergo,netgo,sqlite_omit_load_extension \ -ldflags="-extldflags=-static" [0]: https://www.arp242.net/static-go.html

[deleted]

Re: Go port of SQLite without CGo

#90
post #79

Note that this is not “go port”. It’s C transpiled to go with their custom C compiler, and it’s transpiled differently for each architecture (as it’s using unsafe go heavily) I look at it like a neat exercise but not really something to use in production. (I actually use it in tests though.)

It is definitely neat, but I think it's more than an exercise. I believe he runs it through the extensive SQLite test suite (I've seen that before, though I can't see where that's stated now).

My understanding was that the sqlite test suite is closed source, and kept under pretty tight grip as it represents their competitive advantage to anyone trying to fork sqlite.

Pretty sure the lead dev said something along those lines when he did an episode of Corecursive?

Post reply on HN