Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing a SQLite clone from scratch in C (2017)

cstack.github.io

Post reply on HN
Print view View original

31–40 of 47 posts

Re: Writing a SQLite clone from scratch in C (2017)

#31

Post by [deleted] »

laumars wrote: post #3
melling wrote: post #2

SQLite is already open source in the public domain. https://sqlite.org/src/doc/trunk/README.md Wouldn’t this be more worthwhile to write in [insert favorite modern language]? I get it is to learn. C is a much more difficult language in which to work. A higher level language would allow one to better abstract the concepts and iterate faster

> Wouldn’t this be more worthwhile to write in [insert favorite modern language]? sqlite isn’t your average C development, it’s got more lines of code written for tests than it does for actual code. Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. The only practical reason for rewriting sqlite is…

[deleted]

Re: Writing a SQLite clone from scratch in C (2017)

#32

Post by gizdan »

melling wrote: post #2

SQLite is already open source in the public domain. https://sqlite.org/src/doc/trunk/README.md Wouldn’t this be more worthwhile to write in [insert favorite modern language]? I get it is to learn. C is a much more difficult language in which to work. A higher level language would allow one to better abstract the concepts and iterate faster

There is also a pure go implementation of SQLite[0].

[0] https://gitlab.com/cznic/sqlite

Re: Writing a SQLite clone from scratch in C (2017)

#33

Post by varajelle »

stux wrote: post #30
saurik wrote: post #11

Earlier quoted context omitted.

> Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. And yet, people keep finding use-after-free bugs in sqlite3 that allow attackers to escalate the memory corruption into arbitrary code execution... bugs that have affected major projects, including iCloud and Chrome; here are a handful: there are…

I agree with your general point that "it's very hard to write safe C code", and > people keep finding use-after-free bugs in sqlite3 is true, but > that allow attackers to escalate the memory corruption into arbitrary code execution... bugs that have affected major projects, including iCloud and Chrome; here are a handful: there are lots more even from just the past year :/ is just incorrect. I'd strongly encourage y…

sqlite3 dev: "yes it's an use after free, but it's fine because the attacker don't control SQL query on most applications"

Application dev: "yes it's an SQL injection, but it's fine because this database is only used for unimportant data"

The thing is that the real attacks usually come by chaining a bunch of vulnerabilities together.

Re: Writing a SQLite clone from scratch in C (2017)

#34

Post by varajelle »

correct_horse wrote: post #25
belter wrote: post #19

Earlier quoted context omitted.

I think you make very good points. In your post you also argued for the use of C++, and you commented on how much unhappy you are with the fact Rust has no exceptions. So I take the liberty of asking you a little naughty question :-) What is your take on "Google C++ Style Guide" advice on the use of C++ Exceptions? https://google.github.io/styleguide/cppguide.html "We do not use C++ exceptions" https://google.github.…

I haven't found a good solution to a fallible constructor without exceptions. You might want this if you have a C++ wrapper around a file or some other OS primitive. I would love some allowance for constructors returning std::optional. One problem with this is that child classes' constructors would have to return optional too or else they might throw an exception when they call optional::value.

Don't write faillible constructor. Instead, you can write a static method that returns a std::optional, for example.

Re: Writing a SQLite clone from scratch in C (2017)

#35

Post by [deleted] »

belter wrote: post #19
saurik wrote: post #11

Earlier quoted context omitted.

> Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. And yet, people keep finding use-after-free bugs in sqlite3 that allow attackers to escalate the memory corruption into arbitrary code execution... bugs that have affected major projects, including iCloud and Chrome; here are a handful: there are…

I think you make very good points. In your post you also argued for the use of C++, and you commented on how much unhappy you are with the fact Rust has no exceptions. So I take the liberty of asking you a little naughty question :-) What is your take on "Google C++ Style Guide" advice on the use of C++ Exceptions? https://google.github.io/styleguide/cppguide.html "We do not use C++ exceptions" https://google.github.…

[deleted]

Re: Writing a SQLite clone from scratch in C (2017)

#36

Post by jcelerier »

varajelle wrote: post #34
correct_horse wrote: post #25

Earlier quoted context omitted.

I haven't found a good solution to a fallible constructor without exceptions. You might want this if you have a C++ wrapper around a file or some other OS primitive. I would love some allowance for constructors returning std::optional. One problem with this is that child classes' constructors would have to return optional too or else they might throw an exception when they call optional::value.

Don't write faillible constructor. Instead, you can write a static method that returns a std::optional, for example.

That does not work for data members, unless you are willing to have the entirety of your software wrapped and unwrapped in optionals up to main, which looks pretty much like what people had in the 70s and thought : "okay, maybe there is a language feature we could have to abstract that repetitive mess"

Re: Writing a SQLite clone from scratch in C (2017)

#37

Post by lelanthran »

not2b wrote: post #26
cryo wrote: post #24

Earlier quoted context omitted.

Fair points, although I'd like to keep in mind: Never underestimate a single stubborn person or small team dedicated to an idea. I'm a C++ developer and, for now, sticking to it on larger projects since I love the thinking behind the C++ evolution over the decades. Rust still feels like a hype/moving target. The big selling point appears to be memory safety, but that's just a detail and not enough to justify a switch…

I have spent my career as a C++ developer. What is more interesting to me about Rust isn't memory safety, but safe concurrency. But I have so much legacy code to deal with that all I can do is be influenced by Rust ideas: force all code to document assumptions about ownership and lifetimes, as if we had a borrow checker, and focus attention on redesign of code where this doesn't work cleanly.

> I have spent my career as a C++ developer. What is more interesting to me about Rust isn't memory safety, but safe concurrency.

The safety is limited to data races, not concurrency or parallelism problems.

Re: Writing a SQLite clone from scratch in C (2017)

#38

Post by agent327 »

belter wrote: post #19
saurik wrote: post #11

Earlier quoted context omitted.

> Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. And yet, people keep finding use-after-free bugs in sqlite3 that allow attackers to escalate the memory corruption into arbitrary code execution... bugs that have affected major projects, including iCloud and Chrome; here are a handful: there are…

I think you make very good points. In your post you also argued for the use of C++, and you commented on how much unhappy you are with the fact Rust has no exceptions. So I take the liberty of asking you a little naughty question :-) What is your take on "Google C++ Style Guide" advice on the use of C++ Exceptions? https://google.github.io/styleguide/cppguide.html "We do not use C++ exceptions" https://google.github.…

Googles' ban on exceptions was a historical decision, based on the compilers available at the time. That they would not go back and revise all of their software is understandble: retrofitting exceptions into existing C++ code is painful; identifying the points where a catch handler is needed is typically a hard problem, and retrofitting exception safety anywhere, while always a good idea (even if exceptions aren't used) is a large task if you already have a lot of C++ lying around.

Googles strategy does come with some significant drawbacks, such as requiring init() functions everywhere (what happens if you forget one?), and needing to test error codes after every function call (easy to forget). It also locks them out of useful features like overloaded operators for the most part (since those have no error return option other than exceptions). In general, the 'happy path' becomes cluttered with error handling everywhere, leading to programs that dedicate more lines to error handling than actual processing.

Also, it says "WE do not use exceptions", not "NOBODY should use exceptions". It's a statement about the situation at Google, based on their unique circumstances, not a general guideline.

Re: Writing a SQLite clone from scratch in C (2017)

#39

Post by FBT »

intc wrote: post #27
saurik wrote: post #11

Earlier quoted context omitted.

> Given the extensive tests written against sqlite, rewriting it from scratch in any language, even a more modern/safer one, would ultimately end up with a buggier program. And yet, people keep finding use-after-free bugs in sqlite3 that allow attackers to escalate the memory corruption into arbitrary code execution... bugs that have affected major projects, including iCloud and Chrome; here are a handful: there are…

I personally don't appreciate the idea of using Rust as an overall "only solution" replacement. Requires LLVM (a huge overkill on a system otherwise based on GCC), no proper support for dynamic linking, own build system, absolutely huge "ecosystem", No standardization (-> no competing implementations), SLOW to compile... Yada-yada. Rust it self also has it's fare share of free after use etc bugs. Please check: https:…

> own build system

Nothing in Rust requires you to use Cargo. It's really convenient to have a good build system like Cargo—but if you like C style of manually invoking the compiler, rustc can do that too.

Rust is no worse by bundling Cargo. It strictly dominates the alternative, which would be to just ship rustc and allow the user to pick whatever build system they like. You still can pick your favorite build system; but if you don't have a particularly strong preference, Cargo is a very good default.

Re: Writing a SQLite clone from scratch in C (2017)

#40

Post by enriquto »

melling wrote: post #2

SQLite is already open source in the public domain. https://sqlite.org/src/doc/trunk/README.md Wouldn’t this be more worthwhile to write in [insert favorite modern language]? I get it is to learn. C is a much more difficult language in which to work. A higher level language would allow one to better abstract the concepts and iterate faster

> Wouldn’t this be more worthwhile to write in [insert favorite modern language]?

Only if they want to learn how to write an SQLite clone.

But maybe their goal is to learn C. In that case the SQLite clone is a very good project because there's already huge test suite to test the new implementation.

Post reply on HN