Live data from Hacker News

Writing a SQLite clone from scratch in C (2017)

cstack.github.io

11–20 of 47 posts

Re: Writing a SQLite clone from scratch in C (2017)

#11
post #3
post #2

SQLite is already open source in the public domain. https://sqlite.org/src/doc/trunk/README.md Wouldn’t this be more worthwhile to write in [insert favorite modern language]? I get it is to learn. C is a much more difficult language in which to work. A higher level language would allow one to better abstract the concepts and iterate faster

> Wouldn’t this be more worthwhile to write in [insert favorite modern language]? sqlite isn’t your average C development, it’s got more lines of code written for tests than it does for actual code. Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. The only practical reason for rewriting sqlite is…

> Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program.

And yet, people keep finding use-after-free bugs in sqlite3 that allow attackers to escalate the memory corruption into arbitrary code execution... bugs that have affected major projects, including iCloud and Chrome; here are a handful: there are lots more even from just the past year :/.

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-1363...

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-9327

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-1387...

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-2022...

The reality is that programming in C is just such a horrible thing--making it so easy to accidentally make mistakes that don't merely lead to "your database is corrupted" but escalate all the way to "the attacker is now running their own code on your computer"--that we simply need to stop doing it (preferably 15 years ago: there is really no excuse to not be at least using the C++ compiler to compile all of our C to get at minimum templates, deconstructors, and better type checks), get everyone else to stop doing it, and then actively go through and scour the ecosystem to remove it all, as every line of C running on your computer is a liability.

The underlying thing to appreciate here is that the goal is to make these kinds of mistakes not only much harder but impossible: I simply don't need tests to prove a Rust program (which I say as an example: I am not one of those Rust advocates that you see... I actually hate the language for not having exceptions and I even dislike some of the key people involved in both Rust and Cargo, and so my noting the dominating superiority of Rust to C should come as a pretty strong/sobering commentary; it is unfortunate as I honestly think they are making decisions that have been holding back widespread deployment of Rust, and yet they think they are doing well... hopefully they will claw their way out of the pit soon: at least some progress was finally made recently?) is at least better than sqlite3 apparently is in practice :(.

Re: Writing a SQLite clone from scratch in C (2017)

#12
post #5
post #2

SQLite is already open source in the public domain. https://sqlite.org/src/doc/trunk/README.md Wouldn’t this be more worthwhile to write in [insert favorite modern language]? I get it is to learn. C is a much more difficult language in which to work. A higher level language would allow one to better abstract the concepts and iterate faster

Rewrite in your favorite modern language if you want to make something "better" (or "safer", or some other "-er")... but usually you end up with something worse because usually you don't have the years of trial-and-error, iteration, and refinement that most big name projects have. Rewrite in C if you want to learn the underlying data structures, algorithms, and real-world computer science of the field. Database desig…

[deleted]

Re: Writing a SQLite clone from scratch in C (2017)

#13
post #11
post #3

Earlier quoted context omitted.

> Wouldn’t this be more worthwhile to write in [insert favorite modern language]? sqlite isn’t your average C development, it’s got more lines of code written for tests than it does for actual code. Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. The only practical reason for rewriting sqlite is…

> Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. And yet, people keep finding use-after-free bugs in sqlite3 that allow attackers to escalate the memory corruption into arbitrary code execution... bugs that have affected major projects, including iCloud and Chrome; here are a handful: there are…

You'll have to pry C from my cold, dead hands.

But nobody uses my projects anyways :-/

edit: oh shit, saurik. I respect you work a lot. Still love my insecure "bad" language, though.

Would you have been able to make as big of an impact on securing our digital freedoms without the faults of memory and pointer errors, though? I almost feel like you owe a debt of gratitude to C's faults. I'm half kidding. But only half

Re: Writing a SQLite clone from scratch in C (2017)

#14
post #11
post #3

Earlier quoted context omitted.

> Wouldn’t this be more worthwhile to write in [insert favorite modern language]? sqlite isn’t your average C development, it’s got more lines of code written for tests than it does for actual code. Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. The only practical reason for rewriting sqlite is…

> Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. And yet, people keep finding use-after-free bugs in sqlite3 that allow attackers to escalate the memory corruption into arbitrary code execution... bugs that have affected major projects, including iCloud and Chrome; here are a handful: there are…

It pains me that Rust targets LLVM, and doesn't use C as an intermediary language.

I would _love_ to use Rust on embedded and older hardware on which LLVM simply isn't available.

Re: Writing a SQLite clone from scratch in C (2017)

#15
post #14
post #11

Earlier quoted context omitted.

> Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. And yet, people keep finding use-after-free bugs in sqlite3 that allow attackers to escalate the memory corruption into arbitrary code execution... bugs that have affected major projects, including iCloud and Chrome; here are a handful: there are…

It pains me that Rust targets LLVM, and doesn't use C as an intermediary language. I would _love_ to use Rust on embedded and older hardware on which LLVM simply isn't available.

You might be interested in https://nim-lang.org/ and it's new deterministic memory management method!

Re: Writing a SQLite clone from scratch in C (2017)

#16
post #3
post #2

SQLite is already open source in the public domain. https://sqlite.org/src/doc/trunk/README.md Wouldn’t this be more worthwhile to write in [insert favorite modern language]? I get it is to learn. C is a much more difficult language in which to work. A higher level language would allow one to better abstract the concepts and iterate faster

> Wouldn’t this be more worthwhile to write in [insert favorite modern language]? sqlite isn’t your average C development, it’s got more lines of code written for tests than it does for actual code. Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. The only practical reason for rewriting sqlite is…

> sqlite isn’t your average C development, it’s got more lines of code written for tests than it does for actual code

It’d be a cool exercise to rewrite in a language that easily targets the C ABI and can thus use the same suite of tests for verification. I suspect Zig would be ideal for this, as would maybe Rust.

Edit: looks like the tests are in tcl not C so nevermind!

Re: Writing a SQLite clone from scratch in C (2017)

#17
post #14

Earlier quoted context omitted.

It pains me that Rust targets LLVM, and doesn't use C as an intermediary language. I would _love_ to use Rust on embedded and older hardware on which LLVM simply isn't available.

You might be interested in https://nim-lang.org/ and it's new deterministic memory management method!

Neat! I have tomorrow off so I'll give it a shot.

Re: Writing a SQLite clone from scratch in C (2017)

#18
post #14
post #11

Earlier quoted context omitted.

> Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. And yet, people keep finding use-after-free bugs in sqlite3 that allow attackers to escalate the memory corruption into arbitrary code execution... bugs that have affected major projects, including iCloud and Chrome; here are a handful: there are…

It pains me that Rust targets LLVM, and doesn't use C as an intermediary language. I would _love_ to use Rust on embedded and older hardware on which LLVM simply isn't available.

I'm not sure what value you're suggesting rustc would be able to bring there. If you want that in LLVM you can try your luck with the "resurrected" C backend: https://github.com/JuliaComputingOSS/llvm-cbe

I don't understand why I see so many requests for LLVM-based languages to change around their backend or IR, that seems to be a huge amount of work for comparatively little benefit. The correct thing to do there is to just add support for those to LLVM.

Re: Writing a SQLite clone from scratch in C (2017)

#19
post #11
post #3

Earlier quoted context omitted.

> Wouldn’t this be more worthwhile to write in [insert favorite modern language]? sqlite isn’t your average C development, it’s got more lines of code written for tests than it does for actual code. Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. The only practical reason for rewriting sqlite is…

> Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. And yet, people keep finding use-after-free bugs in sqlite3 that allow attackers to escalate the memory corruption into arbitrary code execution... bugs that have affected major projects, including iCloud and Chrome; here are a handful: there are…

I think you make very good points. In your post you also argued for the use of C++, and you commented on how much unhappy you are with the fact Rust has no exceptions.

So I take the liberty of asking you a little naughty question :-)

What is your take on "Google C++ Style Guide" advice on the use of C++ Exceptions?

https://google.github.io/styleguide/cppguide.html

"We do not use C++ exceptions"

https://google.github.io/styleguide/cppguide.html#Exceptions

Re: Writing a SQLite clone from scratch in C (2017)

#20
post #17

Earlier quoted context omitted.

You might be interested in https://nim-lang.org/ and it's new deterministic memory management method!

Neat! I have tomorrow off so I'll give it a shot.

It's a surprisingly fantastic language. In my opinion, it's the perfect language. But here are some issues in my eyes:

* The community is tiny. A rounding error compared to Go, Rust, etc.

* The core team of contributors is even smaller. You could comfortably have an intimate dinner party with all of them. If they lose interest, the language would likely die.

* The library distribution story wasn't great last I checked.

Caveat: I haven't used Nim for more than a year now. Maybe all these issues are no longer relevant. I really want it to succeed, but who knows what the future holds.

Post reply on HN