Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Is SQLite Coded In C

sqlite.org

Post reply on HN
Print view View original

91–100 of 411 posts

Re: Why Is SQLite Coded In C

#91

Post by btilly »

jonahx wrote: post #34
bfkwlfkjf wrote: post #16

> Safe languages insert additional machine branches to do things like verify that array accesses are in-bounds. In correct code, those branches are never taken. That means that the machine code cannot be 100% branch tested, which is an important component of SQLite's quality strategy. Huh it's not everyday that I hear a genuinely new argument. Thanks for sharing.

So is the argument that safe langs produce stuff like: // pseudocode if (i >= array_length) panic("index out of bounds") that are never actually run if the code is correct? But (if I understand correctly) these are checks implicitly added by the compiler. So the objection amounts to questioning the correctness of this auto-generated code, and is predicated upon mistrusting the correctness of the compiler? But presuma…

One of the things that SQLite is explicitly designed to do is have predictable behavior in a lot of conditions that shouldn't happen. One of those predictable behavior is that it does its best to stay up and running, and continuing to do the best it can. Conditions where it should succeed in doing this include OOM, the possibility of corrupted data files, and (if possible) misbehaving CPUs.

Automatic array bounds checks can get hit by corrupted data. Thereby leading to a crash of exactly the kind that SQLite tries to avoid. With complete branch testing, they can guarantee that the test suite includes every kind of corruption that might hit an array bounds check, and guarantee that none of them panic. But if the compiler is inserting branches that are supposed to be inaccessible, you can't do complete branch testing. So now how do you know that you have tested every code branch that might be reached from corrupted data?

Furthermore those unused branches are there as footguns which are reachable with a cosmic ray bit flip, or a dodgy CPU. Which again undermines the principle of keeping running if at all possible.

Re: Why Is SQLite Coded In C

#92

Post by gerdesj »

steveklabnik wrote: post #27
DarkNova6 wrote: post #19

> All that said, it is possible that SQLite might one day be recoded in Rust. Recoding SQLite in Go is unlikely since Go hates assert(). But Rust is a possibility. Some preconditions that must occur before SQLite is recoded in Rust include: - Rust needs to mature a little more, stop changing so fast, and move further toward being old and boring. - Rust needs to demonstrate that it can be used to create general-purpos…

1. Rust has had ten years since 1.0. It changes in backward compatible ways. For some people, they want no changes at all, so it’s important to nail down which sense is meant. 2. This has been demonstrated. 3. This one hinges on your definition of “obscure,” but the “without an operating system” bit is unambiguously demonstrated. 4. I am not an expert here, but given that you’re testing binaries, I’m not sure what is…

"1. Rust has had ten years since 1.0. ..."

Rust insists on its own package manager "rustup" and frowns on distro maintainers. When Rust is happy to just be packaged by the distro and rustup has gone away, then it will have matured to at least adolescence.

Re: Why Is SQLite Coded In C

#93

Post by daxfohl »

It sounds like the core doesn't even allocate, and presumably the extended library allocates in limited places using safe patterns. So there wouldn't be much benefit from Rust anyway, I'd think. Had SQLite ever had a memory leak or use-after-delete bug on a production release? If so, that answers the question. But I've never heard of one.

Also, does it use doubly linked lists or graphs at all? Those can, in a way, be safer in C since Rust makes you roll your own virtual pointer arena.

Re: Why Is SQLite Coded In C

#94

Post by estebank »

steveklabnik wrote: post #53
jokoon wrote: post #43

I wonder if the hype helps rust being a better language At this point I wish the creators of the language could talk about what rust is bad at.

Folks involved often do! Talking about what’s not great is the only path towards getting better, because you have to identify pain points in order to fix them.

I would go as far as saying that 90% of managing the project is properly communicating, discussing and addressing the ways in which Rust sucks. The all-hands in NL earlier this year was wall to wall meetings about how much things suck and what to do about them! I mean this in the best possible way. ^_^

Re: Why Is SQLite Coded In C

#95

Post by refulgentis »

estebank wrote: post #87
Aurornis wrote: post #62

Earlier quoted context omitted.

I guess I don’t find that argument very compelling. If you’re convinced the code branch can’t ever be taken, you also should be confident that it doesn’t need to be tested. This feels like chasing arbitrary 100% test coverage at the expense of safety. The code quality isn’t actually improved by omitting the checks even though it makes testing coverage go up.

In safety critical spaces you need to be able to trace any piece of a binary back to code back to requirements. If a piece of running code is implicit in code, it makes that traceability back to requirements harder. But I'd be surprised if things like bounds checks are really a problem for that kind of analysis.

Yeah sounds too clever by half, memory safe languages are less safe because they have bounds checks...maybe I could see it on a space shuttle? Well, only in the most CYA scenarios, I'd imagine.

Re: Why Is SQLite Coded In C

#96

Post by foul »

selcuka wrote: post #67
lionkor wrote: post #50

Earlier quoted context omitted.

It's not like that, the compiler explicitly doesn't do compile-time checks here and offloads those to the runtime. Rust does not stop you from writing code that accesses out of bounds, at all. It just makes sure that there's an if that checks.

Ok, but you can still test all the branches in your source code and have 100% coverage. Those additional `if` branches are added by the compiler. You are responsible for testing the code you write, not the one that actually runs. Your compiler's test suite is responsible for the rest. By the same logic one could also claim that tail recursion optimisation, or loop unrolling are also dangerous because they change the…

>You are responsible for testing the code you write, not the one that actually runs.

Hipp worked as a military contractor for battleships, furthermore years later SQLite was under contract under every proto-smartphone company in the USA. Under these constraints you maybe are not responsible to test what the compiler spits out across platforms and different compilers, but doing that makes the project a lot more reliable, makes it sexier for embedded and weapons.

Re: Why Is SQLite Coded In C

#97

Post by dathinab »

bfkwlfkjf wrote: post #16

> Safe languages insert additional machine branches to do things like verify that array accesses are in-bounds. In correct code, those branches are never taken. That means that the machine code cannot be 100% branch tested, which is an important component of SQLite's quality strategy. Huh it's not everyday that I hear a genuinely new argument. Thanks for sharing.

the problem is it's kinda an anti argument

you basically say if deeply unexpected things happen you prefer you program doing widely arbitrary and as such potentially dangerous things over it having a clean abort or proper error. ... that doesn't seem right

worse it's due to a lack of the used tooling and not a fundamental problem, not only can you test this branches (using fault injection) you also often (not always) can separate them from relevant branches when collecting the branch statistics

so the while argument misses the point (which is tooling is lacking, not extra checks for array bounds and similar)

lastly array bounds checking is probably the worst example they could have given as it

- often can be disabled/omitted in optimized builds

- is quite often optimized away

- has often quite low perf. overhead

- bound check branches are often very easy to identify, i.e. excluding them from a 100% branch testing statistic is viable

- out of bounds read/write are some of the most common cases of memory unsafety leading to security vulnerability (including full RCE cases)

Re: Why Is SQLite Coded In C

#98

Post by turtletontine »

saalweachter wrote: post #51

I think beyond the historical reasons why C was the best choice when SQLite was being developed, or the advantages it has today, there's also just no reason to rewrite SQLite in another language . We don't have to have one implementation of a lightweight SQL database. You can go out right now and start your own implementation in Rust or C++ or Go or Lisp or whatever you like! You can even make compatible APIs for it…

Thanks for this, I fully agree. One frustration I have with the modern moment is the tendency to view anything more than five years old with disdain, as utterly irrelevant and obsolete. Maybe I’m just going old, but I like my technology dependable and boring, especially software. Glad to see someone express respect for the decades of expertise that have gone into things we take for granted.

Re: Why Is SQLite Coded In C

#99

Post by [deleted] »

bfkwlfkjf wrote: post #16

> Safe languages insert additional machine branches to do things like verify that array accesses are in-bounds. In correct code, those branches are never taken. That means that the machine code cannot be 100% branch tested, which is an important component of SQLite's quality strategy. Huh it's not everyday that I hear a genuinely new argument. Thanks for sharing.

[deleted]

Re: Why Is SQLite Coded In C

#100

Post by accelbred »

ChadNauseam wrote: post #32
bfkwlfkjf wrote: post #16

> Safe languages insert additional machine branches to do things like verify that array accesses are in-bounds. In correct code, those branches are never taken. That means that the machine code cannot be 100% branch tested, which is an important component of SQLite's quality strategy. Huh it's not everyday that I hear a genuinely new argument. Thanks for sharing.

I wonder if this problem could be mitigated by not requiring coverage of branches that unconditionally lead to panics. or if there could be some kind of marking on those branches that indicate that they should never occur in correct code

You'd want to statically prove that any panic is unreachable
Post reply on HN