Writing a SQLite clone from scratch in C (2017) • Hacker News Board

Re: Writing a SQLite clone from scratch in C (2017)

#21

Post by sidlls » Sun, Jul 04, 2021, 10:38 PM UTC

belter wrote: post #19

saurik wrote: post #11

Earlier quoted context omitted.

> Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. And yet, people keep finding use-after-free bugs in sqlite3 that allow attackers to escalate the memory corruption into arbitrary code execution... bugs that have affected major projects, including iCloud and Chrome; here are a handful: there are…

I think you make very good points. In your post you also argued for the use of C++, and you commented on how much unhappy you are with the fact Rust has no exceptions. So I take the liberty of asking you a little naughty question :-) What is your take on "Google C++ Style Guide" advice on the use of C++ Exceptions? https://google.github.io/styleguide/cppguide.html "We do not use C++ exceptions" https://google.github.…

That Google does it isn’t sufficient evidence that it’s good.

Re: Writing a SQLite clone from scratch in C (2017)

#22

Post by owl57 » Sun, Jul 04, 2021, 11:21 PM UTC

sidlls wrote: post #21

belter wrote: post #19

Earlier quoted context omitted.

I think you make very good points. In your post you also argued for the use of C++, and you commented on how much unhappy you are with the fact Rust has no exceptions. So I take the liberty of asking you a little naughty question :-) What is your take on "Google C++ Style Guide" advice on the use of C++ Exceptions? https://google.github.io/styleguide/cppguide.html "We do not use C++ exceptions" https://google.github.…

That Google does it isn’t sufficient evidence that it’s good.

In this particular section, the authors already took good care to separate internal rules from general advice: "On their face, the benefits of using exceptions outweigh the costs, especially in new projects. … Things would probably be different if we had to do it all over again from scratch."

Re: Writing a SQLite clone from scratch in C (2017)

#23

Post by belter » Sun, Jul 04, 2021, 11:39 PM UTC

owl57 wrote: post #22

sidlls wrote: post #21

Earlier quoted context omitted.

That Google does it isn’t sufficient evidence that it’s good.

In this particular section, the authors already took good care to separate internal rules from general advice: "On their face, the benefits of using exceptions outweigh the costs, especially in new projects. … Things would probably be different if we had to do it all over again from scratch."

You are correct and I did not overlook that part.

Also as stated in the guidelines, some of the reasons are the intention of integrating the open source projects with an internal Google C++ code base that does not use exceptions. So integration would be difficult.

However, the question is how come such a large code base using C++ without Exceptions come to be. Specially on a company with so many employees chairs of C++ committees :-)

Just to make it clear, I prefer the C and Rust model around error handling. I will also concede that, for GUI applications and when on a coherent code base, you probably want C++ Exceptions.

Re: Writing a SQLite clone from scratch in C (2017)

#24

Post by cryo » Sun, Jul 04, 2021, 11:40 PM UTC

pugz wrote: post #20

dleslie wrote: post #17

Earlier quoted context omitted.

Neat! I have tomorrow off so I'll give it a shot.

It's a surprisingly fantastic language. In my opinion, it's the perfect language. But here are some issues in my eyes: * The community is tiny. A rounding error compared to Go, Rust, etc. * The core team of contributors is even smaller. You could comfortably have an intimate dinner party with all of them. If they lose interest, the language would likely die. * The library distribution story wasn't great last I checke…

Fair points, although I'd like to keep in mind: Never underestimate a single stubborn person or small team dedicated to an idea.

I'm a C++ developer and, for now, sticking to it on larger projects since I love the thinking behind the C++ evolution over the decades. Rust still feels like a hype/moving target. The big selling point appears to be memory safety, but that's just a detail and not enough to justify a switch. I'm keeping an eye on how Rust and Nim will evolve in the next 5–10 years, but for now, C++, Erlang, and Lisp are my go-to choices for projects which should last the upcoming decades.

Re: Writing a SQLite clone from scratch in C (2017)

#25

Post by correct_horse » Mon, Jul 05, 2021, 12:10 AM UTC

belter wrote: post #19

saurik wrote: post #11

Earlier quoted context omitted.

> Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. And yet, people keep finding use-after-free bugs in sqlite3 that allow attackers to escalate the memory corruption into arbitrary code execution... bugs that have affected major projects, including iCloud and Chrome; here are a handful: there are…

I think you make very good points. In your post you also argued for the use of C++, and you commented on how much unhappy you are with the fact Rust has no exceptions. So I take the liberty of asking you a little naughty question :-) What is your take on "Google C++ Style Guide" advice on the use of C++ Exceptions? https://google.github.io/styleguide/cppguide.html "We do not use C++ exceptions" https://google.github.…

I haven't found a good solution to a fallible constructor without exceptions. You might want this if you have a C++ wrapper around a file or some other OS primitive. I would love some allowance for constructors returning std::optional. One problem with this is that child classes' constructors would have to return optional too or else they might throw an exception when they call optional::value.

Re: Writing a SQLite clone from scratch in C (2017)

#26

Post by not2b » Mon, Jul 05, 2021, 1:10 AM UTC

cryo wrote: post #24

pugz wrote: post #20

Earlier quoted context omitted.

It's a surprisingly fantastic language. In my opinion, it's the perfect language. But here are some issues in my eyes: * The community is tiny. A rounding error compared to Go, Rust, etc. * The core team of contributors is even smaller. You could comfortably have an intimate dinner party with all of them. If they lose interest, the language would likely die. * The library distribution story wasn't great last I checke…

Fair points, although I'd like to keep in mind: Never underestimate a single stubborn person or small team dedicated to an idea. I'm a C++ developer and, for now, sticking to it on larger projects since I love the thinking behind the C++ evolution over the decades. Rust still feels like a hype/moving target. The big selling point appears to be memory safety, but that's just a detail and not enough to justify a switch…

I have spent my career as a C++ developer. What is more interesting to me about Rust isn't memory safety, but safe concurrency. But I have so much legacy code to deal with that all I can do is be influenced by Rust ideas: force all code to document assumptions about ownership and lifetimes, as if we had a borrow checker, and focus attention on redesign of code where this doesn't work cleanly.

Re: Writing a SQLite clone from scratch in C (2017)

#27

Post by intc » Mon, Jul 05, 2021, 1:28 AM UTC

saurik wrote: post #11

laumars wrote: post #3

Earlier quoted context omitted.

> Wouldn’t this be more worthwhile to write in [insert favorite modern language]? sqlite isn’t your average C development, it’s got more lines of code written for tests than it does for actual code. Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. The only practical reason for rewriting sqlite is…

> Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. And yet, people keep finding use-after-free bugs in sqlite3 that allow attackers to escalate the memory corruption into arbitrary code execution... bugs that have affected major projects, including iCloud and Chrome; here are a handful: there are…

I personally don't appreciate the idea of using Rust as an overall "only solution" replacement. Requires LLVM (a huge overkill on a system otherwise based on GCC), no proper support for dynamic linking, own build system, absolutely huge "ecosystem", No standardization (-> no competing implementations), SLOW to compile... Yada-yada.

Rust it self also has it's fare share of free after use etc bugs. Please check: https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=Rust

I'm personally thankful that we have SQLite which is written in C - It's efficient, tiny and severe bugs are relatively rare.

As the original article quotes: “What I cannot create, I do not understand.”

Re: Writing a SQLite clone from scratch in C (2017)

#28

Post by timClicks » Mon, Jul 05, 2021, 2:34 AM UTC

intc wrote: post #27

saurik wrote: post #11

Earlier quoted context omitted.

> Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. And yet, people keep finding use-after-free bugs in sqlite3 that allow attackers to escalate the memory corruption into arbitrary code execution... bugs that have affected major projects, including iCloud and Chrome; here are a handful: there are…

I personally don't appreciate the idea of using Rust as an overall "only solution" replacement. Requires LLVM (a huge overkill on a system otherwise based on GCC), no proper support for dynamic linking, own build system, absolutely huge "ecosystem", No standardization (-> no competing implementations), SLOW to compile... Yada-yada. Rust it self also has it's fare share of free after use etc bugs. Please check: https:…

> Requires LLVM (a huge overkill on a system otherwise based on GCC), no proper support for dynamic linking, own build system, absolutely huge "ecosystem", No standardization (-> no competing implementations), SLOW to compile.

All of those are secondary if Rust is able to reliably produce memory safe software.

> Rust it self also has it's fare share of free after use etc bugs. Please check: https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=Rust

A relatively quick skim through those issues seems to indicate that the unsafe keyword really is unsafe. I wonder if SQLite can be written without it Rust?

Re: Writing a SQLite clone from scratch in C (2017)

#29

Post by kahlonel » Mon, Jul 05, 2021, 2:57 AM UTC

saurik wrote: post #11

laumars wrote: post #3

Earlier quoted context omitted.

> Wouldn’t this be more worthwhile to write in [insert favorite modern language]? sqlite isn’t your average C development, it’s got more lines of code written for tests than it does for actual code. Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. The only practical reason for rewriting sqlite is…

> Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. And yet, people keep finding use-after-free bugs in sqlite3 that allow attackers to escalate the memory corruption into arbitrary code execution... bugs that have affected major projects, including iCloud and Chrome; here are a handful: there are…

> The underlying thing to appreciate here is that the goal is to make these kinds of mistakes not only much harder but impossible

Hate to break it to you, but it is not "impossible" to have CVEs in Rust [1].

1. https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=rust

Re: Writing a SQLite clone from scratch in C (2017)

#30

Post by stux » Mon, Jul 05, 2021, 3:18 AM UTC

saurik wrote: post #11

laumars wrote: post #3

Earlier quoted context omitted.

> Wouldn’t this be more worthwhile to write in [insert favorite modern language]? sqlite isn’t your average C development, it’s got more lines of code written for tests than it does for actual code. Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. The only practical reason for rewriting sqlite is…

> Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. And yet, people keep finding use-after-free bugs in sqlite3 that allow attackers to escalate the memory corruption into arbitrary code execution... bugs that have affected major projects, including iCloud and Chrome; here are a handful: there are…

I agree with your general point that "it's very hard to write safe C code", and

> people keep finding use-after-free bugs in sqlite3

is true, but

> that allow attackers to escalate the memory corruption into arbitrary code execution... bugs that have affected major projects, including iCloud and Chrome; here are a handful: there are lots more even from just the past year :/

is just incorrect. I'd strongly encourage you to read https://www.sqlite.org/cves.html