Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Is SQLite Coded In C

sqlite.org

Post reply on HN
Print view View original

71–80 of 411 posts

Re: Why Is SQLite Coded In C

#71

Post by a-dub »

pella wrote: post #13
pm2222 wrote: post #6

These points strike me: Safe languages insert additional machine branches to do things like verify that array accesses are in-bounds. In correct code, those branches are never taken. That means that the machine code cannot be 100% branch tested, which is an important component of SQLite's quality strategy. Rust needs to mature a little more, stop changing so fast, and move further toward being old and boring. Rust ne…

Turso: https://algora.io/challenges/turso "Turso is rewriting SQLite in Rust ; Find a bug to win $1,000" ------ - Dec 10, 2024 : "Introducing Limbo: A complete rewrite of SQLite in Rust" https://turso.tech/blog/introducing-limbo-a-complete-rewrite... - Jan 21, 2025 - "We will rewrite SQLite. And we are going all-in" https://turso.tech/blog/we-will-rewrite-sqlite-and-we-are-go... - Project: https://github.com/tursodat…

sqlite3 has one (apparently this is called "the amalgamation") c source file that is ~265 kloc (!) long with external dependencies on zlib, readline and ncurses. built binaries are libsqlite3.so at 4.8M and sqlite3 at 6.1M.

turso has 341 rust source files spread across tens of directories and 514 (!) external dependencies that produce (in release mode) 16 libraries and 7 binaries with tursodb at 48M and libturso_sqlite3.so at 36M.

looks roughly an order of magnitude larger to me. it would be interesting to understand the memory usage characteristics in real-world workloads. these numbers also sort of capture the character of the languages. for extreme portability and memory efficiency, probably hard to beat c and autotools though.

Re: Why Is SQLite Coded In C

#72

Post by lifthrasiir »

sema4hacker wrote: post #59

"Why SQLite is coded in C..." is an explanation, as documented at sqlite.org. "Why is SQLite coded in C and not Rust?" is a question, which immediately makes me want to ask "Why do you need SQLite coded in Rust?".

Because the title has been editorialized.

Re: Why Is SQLite Coded In C

#73

Post by Cloudef »

metaltyphoon wrote: post #45
Cloudef wrote: post #36

Earlier quoted context omitted.

I think zig generally composes better than rust. With rust you pretty much have to start over if you want reusable / composable code, that is not use the default std. Rust has small crates for every little thing because it doesn't compose well, as well to improve compile times. libc in the default std also is major L.

> I think zig generally composes better than rust. I read your response 3 times and I truly don't know what you mean. Mind explaining with a simple example?

It mainly comes down how the std is designed. Zig has many good building blocks like allocators, and how every function that allocates something takes one. This allows you to reuse the same code for different kind of situations.

Hash maps in zig std are another great example, where you can use adapter to completely change how the data is stored and accessed while keeping the same API [1]. For example to have map with limited memory bound that automatically truncates itself, in rust you need to either write completely new data structure for this or rely on someone's crate again (indexmap).

Errors in zig compose also better, in rust I find error handling really annoying. Anyhow makes it better for application development but you shouldn't use it if writing libraries.

When writing zig I always feel like I can reuse pieces of existing code by combining the building blocks at hand (including freestanding targets!). While in rust I always feel like you need go for the fully tailored solution with its own gotchas, which is ironic considering how many crates there are and how many crates projects depend on vs. typical zig projects that often don't depend on lots of stuff.

1: https://zig.news/andrewrk/how-to-use-hash-map-contexts-to-sa...

Re: Why Is SQLite Coded In C

#74

Post by urbandw311er »

sema4hacker wrote: post #59

"Why SQLite is coded in C..." is an explanation, as documented at sqlite.org. "Why is SQLite coded in C and not Rust?" is a question, which immediately makes me want to ask "Why do you need SQLite coded in Rust?".

Indeed. Why is SQLite coded in C and not BASIC?

Re: Why Is SQLite Coded In C

#75

Post by unclad5968 »

sedatk wrote: post #40
steveklabnik wrote: post #12

Earlier quoted context omitted.

If the branch is never taken, and the optimizer can prove it, it will remove the check. Sometimes if it can’t actually prove it there’s ways to help it understand, or, in the almost extreme case, you do what I commented below.

Yeah I don't understand the argument. If you can't convince the compiler that that branch will never be taken, then I strongly suspect that it may be taken.

That's not the point. The point is that if it is never taken, you can't test it. They don't care that it inserts a conditional OP to check, they care that they can't test the conditional path.

Re: Why Is SQLite Coded In C

#76

Post by daxfohl »

blibble wrote: post #58
slashdev wrote: post #31

This is ignoring the elephant in the room: SQLite is being rewritten in Rust and it's going quite well. https://github.com/tursodatabase/turso It has async I/O support on Linux with io_uring, vector support, BEGIN CONCURRENT for improved write throughput using multi-version concurrency control (MVCC), Encryption at rest, incremental computation using DBSP for incremental view maintenance and query subscriptions. Time…

> Time will tell, but this may well be the future of SQLite. turdso is VC funded so will probably be defunct in 2 years

Or, so it's being written mostly by AI.

Re: Why Is SQLite Coded In C

#77

Post by NobodyNada »

bfkwlfkjf wrote: post #16

> Safe languages insert additional machine branches to do things like verify that array accesses are in-bounds. In correct code, those branches are never taken. That means that the machine code cannot be 100% branch tested, which is an important component of SQLite's quality strategy. Huh it's not everyday that I hear a genuinely new argument. Thanks for sharing.

It's interesting to consider (and the whole page is very well-reasoned), but I don't think that the argument holds up to scrutiny. If such an automatic bounds-check fails, then the program would have exhibited undefined behavior without that branch -- and UB is strictly worse than an unreachable branch that does something well-specified like aborting.

A simple array access in C:

    arr[i] = 123;
...can be thought of as being equivalent to:

    if (i >= array_length) UB();
    else arr[i] = 123;
where the "UB" function can do literally anything. From the perspective of exhaustively testing and formally verifying software, I'd rather have the safe-language equivalent:

    if (i >= array_length) panic();
    else arr[i] = 123;
...because at least I can reason about what happens if the supposedly-unreachable condition occurs.

Dr. Hipp mentions that "Recoding SQLite in Go is unlikely since Go hates assert()", implying that SQLite makes use of assert statements to guard against unreachable conditions. Surely his testing infrastructure must have some way of exempting unreachable assert branches -- so why can't bounds checks (that do nothing but assert undefined behavior does not occur) be treated in the same way?

Re: Why Is SQLite Coded In C

#78

Post by steveklabnik »

dgfitz wrote: post #63
steveklabnik wrote: post #28

Earlier quoted context omitted.

Other than some operating systems projects, I haven’t run into a “requires nightly” in the wild for years. Most users use the stable releases. (There are some decent reasons to use the nightly toolchain in development even if you don’t rely on any unfinished features in your codebase, but that means they build on stable anyway just fine if you prefer.)

Good to know, maybe I’ll give it a whirl. I’d been under the (mistaken, apparently) impression that if one didn’t update monthly they were going to have a bad time.

You may be running into forwards compatibility issues, not backwards compatibility issues, which is what nightly is about.

The Rust Project releases a new stable compiler every six weeks. Because it is backwards compatible, most people update fairly quickly, as it is virtually always painless. So this may mean, if you don’t update your compiler, you may try out a new package version and it may use features or standard library calls that don’t exist in the version you’re using, because the authors updated regularly. There’s been some developments in Cargo to try and mitigate some of this, but since it’s not what the majority of users do, it’s taken a while and those features landed relatively recently, so they’re not widely adopted yet.

Nightly features are ones that aren’t properly accepted into the language yet, and so are allowed to break in backwards incompatible ways at any time.

Re: Why Is SQLite Coded In C

#79

Post by Havoc »

For a project that is functionally “done” switching doesn’t make sense. Something like kernel code where you know it’ll continue to evolve - there going through the pain may be worth it

Re: Why Is SQLite Coded In C

#80

Post by sedatk »

unclad5968 wrote: post #75
sedatk wrote: post #40

Earlier quoted context omitted.

Yeah I don't understand the argument. If you can't convince the compiler that that branch will never be taken, then I strongly suspect that it may be taken.

That's not the point. The point is that if it is never taken, you can't test it. They don't care that it inserts a conditional OP to check, they care that they can't test the conditional path.

But, there is no conditional path when the type system can assure the compiler that there is nothing to be conditional about. Do they mean that it's impossible to be 100% sure about if there's a conditional path or not?
Post reply on HN