Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I benchmarked six Go SQLite drivers

github.com

Post reply on HN
Print view View original

41–50 of 64 posts

Re: I benchmarked six Go SQLite drivers

#41

Post by morelisp »

Traubenfuchs wrote: post #5

What a curious state, there is just one jdbc (=java) driver for SQLite. Why are there 6 (or more!) for Go?

Aside from the threading mismatch,

- There is a version transpiled from C to Go. A transpilation from C to Java (or even the JVM) would be considerably more difficult, a naive translation would likely undergo a much larger performance hit. Maybe with Valhalla this will change.

- Modern Java deployment is not generally complicated by JNI/JNA (it's already that complicated to start with; Maven already wrangles some of it, although it's still a pain in many cases). Go deployments are simpler if no C linkage is involved.

- Some of these are not database/sql, the equivalent of JDBC, drivers. They're purpose-built drivers that expose rich SQLite-specific features. With the huge popularity of Spring most developers don't even interact at the JDBC level today, only e.g. JPA. IMO Java developers are missing out on richer SQL features in their DBs, but well, they seem to mostly manage.

Re: I benchmarked six Go SQLite drivers

#42

Post by morelisp »

butlerm wrote: post #35
lmz wrote: post #11

Earlier quoted context omitted.

Also because there's a mismatch between goroutines and C threads as described here https://www.cockroachlabs.com/blog/the-cost-and-complexity-o... while Java threads can map 1:1 to C threads.

Perhaps someone should define a new C compatible threading API to allow C libraries (including glibc or a wrapper around glibc) to work with something other than native pthreads. Such as goroutines or Java threads and so on.

Many general M:N threading solutions have been tried over the years. As far as I know current thinking is still that you need substantial cooperation from a language runtime to make it worthwhile. (And even then it's hard - Java's first attempt failed and they went 1:1 essentially between 1998-2022.)

Re: I benchmarked six Go SQLite drivers

#43

Post by neonsunset »

There was a comment from https://news.ycombinator.com/user?id=HackerThemAll with neutral rating that was likely taken down by a mod action despite not violating the rules.

I am reposting it below shortly with minor stylistic changes for politeness.

“Go's SQLite drivers exhibit surprisingly poor performance. In C# I was able to conduct 2 millions of point queries per second on my laptop. And it's not the fastest language in the world.

Go is overrated. It's crudely trying to imitate what Pascal had in '80s using awkward syntax and tooling, but giving you extra CVEs for free.”

Personally, I did not expect it to be this bad…and C# SQLite drivers aren’t even something new - most of them have been written eons ago and consist of fairly standard somewhat allocatey code.

I wonder if it’s because of significant interop overhead in Go, or just fundamental language limitations and quality issues.

Re: I benchmarked six Go SQLite drivers

#44

Post by erik_seaberg »

morelisp wrote: post #41
Traubenfuchs wrote: post #5

What a curious state, there is just one jdbc (=java) driver for SQLite. Why are there 6 (or more!) for Go?

Aside from the threading mismatch, - There is a version transpiled from C to Go. A transpilation from C to Java (or even the JVM) would be considerably more difficult, a naive translation would likely undergo a much larger performance hit. Maybe with Valhalla this will change. - Modern Java deployment is not generally complicated by JNI/JNA (it's already that complicated to start with; Maven already wrangles some of…

For light integration testing, Apache Derby has pretty good fidelity with prod SQL databases. Never benched it but perf wasn’t a problem.

Re: I benchmarked six Go SQLite drivers

#46

Post by zombiezen »

acatton wrote: post #12

In my personal opinion and usage, the performance doesn't matter. Only one driver is written in pure go, and can be easily statically compiled and/or cross-compiled. > modernc, modernc.org/sqlite, a pure Go solution. This is a newer library, based on the SQLite C code re-written in Go. Unless I'm mistaken, this is not a re-write in Go. This is a transpilation of the the SQLite C library into go, using https://gitlab.…

zombiezen.com/go/sqlite is built on top of modernc.org/sqlite, so also has the same properties.

Re: I benchmarked six Go SQLite drivers

#47

Post by justinclift »

As a data point, the gwenn/gosqlite one for GitHub returns field data correctly even when an individual fields' data type doesn't match the column definition.

https://github.com/gwenn/gosqlite

That's important when processing data from untrusted sources (user generated content, etc).

No idea how it compares to the others performance wise though. :)

Re: I benchmarked six Go SQLite drivers

#49

Post by tedunangst »

acatton wrote: post #12

In my personal opinion and usage, the performance doesn't matter. Only one driver is written in pure go, and can be easily statically compiled and/or cross-compiled. > modernc, modernc.org/sqlite, a pure Go solution. This is a newer library, based on the SQLite C code re-written in Go. Unless I'm mistaken, this is not a re-write in Go. This is a transpilation of the the SQLite C library into go, using https://gitlab.…

Have you tested that the cross-compiled pure go sqlite runs, not just compiles, on every platform? (Spoiler: it does not.)
Post reply on HN