In case anyone else was wondering, SQLite is about 156k lines of code with 92,000k lines of test code.
Do you have a source for that?
Limbo: A complete rewrite of SQLite in Rust
171–180 of 238 posts
Re: Limbo: A complete rewrite of SQLite in Rust
#172I'm not buying the rationale in the "async IO" section.
First, there's no need to rewrite anything to add an async interface to sqlite if you want (many clients do, whether local or remote).
The issue with sqlite's synchronous interface is leaving a thread idle while you wait for IO. But I wonder how much of an issue that really is. sqlite is designed to run very locally to the storage, and can make use of native file caching, etc, which makes IO blocking very short if not zero. You wonder if applications have enough idling sqlite threads to justify the switching. (It's not free and would be at quite a fine-grained level.)
The section does mention remote storage, but in that case you're much better off with an async client talking to compute running sqlite, sync interface and all, that is very local to the storage. AKA, a client/server database.
Also, in the WASM section, we're still talking about something that would best be implemented as a sqlite client/wrapper, with no need at all to rewrite it.
Re: Limbo: A complete rewrite of SQLite in Rust
#173> SQLite’s test suite is proprietary This is literally the first time I've ever heard of this, for any project anywhere. I suppose Android is built a bit in this way, but that's a whole other can of worms.
They have a test suite that is part of SQLite3 then public domain product, and they have a much bigger and better test suite that is proprietary.
Re: Limbo: A complete rewrite of SQLite in Rust
#174> SQLite’s test suite is proprietary This is literally the first time I've ever heard of this, for any project anywhere. I suppose Android is built a bit in this way, but that's a whole other can of worms.
Well, Java during its initial life was controlled to a degree through the control of the tests: https://en.wikipedia.org/wiki/Technology_Compatibility_Kit .
Huh, I wasn't aware that Java was initially open source
Re: Limbo: A complete rewrite of SQLite in Rust
#175Re: Limbo: A complete rewrite of SQLite in Rust
#176> SQLite’s test suite is proprietary This is literally the first time I've ever heard of this, for any project anywhere. I suppose Android is built a bit in this way, but that's a whole other can of worms.
Large chunks of the test suite are open source, committed to the repo and easy to run with a `make test`.
Everytime a bug is reported in the forums, the open source tests are updated as part of the bug fix for everyone to see.
There's a separate test suite that offers 100% coverage, that is proprietary, and which was created for certification for use in safety critical environments.
HN loves to discuss business models for open source, but apparently has a problem with this one. Why?
Re: Limbo: A complete rewrite of SQLite in Rust
#177I'm not buying the rationale in the "async IO" section. First, there's no need to rewrite anything to add an async interface to sqlite if you want (many clients do, whether local or remote). The issue with sqlite's synchronous interface is leaving a thread idle while you wait for IO. But I wonder how much of an issue that really is. sqlite is designed to run very locally to the storage, and can make use of native fil…
> The issue with sqlite's synchronous interface is leaving a thread idle while you wait for IO
That's not the only issue. waiting for the result of every read to be able to queue the next read is also an issue, particularily for a VFS that exists on a network (which is a target of theirs, they explicitly mention S3).
I'm not sure if they also are doing work on improving this, but I'm sure that theoretically many reads and writes that SQLite does do not depend on all previous reads and writes, which means you could queue many of them earlier. If your latency to storage is large, this can be a huge performance difference.
Re: Limbo: A complete rewrite of SQLite in Rust
#178> SQLite’s test suite is proprietary This is literally the first time I've ever heard of this, for any project anywhere. I suppose Android is built a bit in this way, but that's a whole other can of worms.
It could be simply to prevent forks, but if it really is 100% branch coverage, why do they still have memory safety related CVE coming out? With asan turned on, and full static analysis, that should make such errors exceedingly rare. Part of the benefit of rust is that it makes coverage both easier to get due to its type system, and less necessary because of the guarantees it makes. But if they really went all the wa…
They claim 100% on https://en.wikipedia.org/wiki/Modified_condition/decision_co...
However, unless you can guarantee that every branch tested has been covered for all possibly relevant application states, that does not preclude CVEs.
Re: Limbo: A complete rewrite of SQLite in Rust
#179Earlier quoted context omitted.
What's wrong with public domain?
Public domain rights differ across countries. So someone could put a work into the public domain and sue you in the one or two major countries where copyright can’t be disclaimed by the original author without a license, or where there isn’t legal precedent for it (this is purely theoretical and has never actually occurred before, it would also not be likely to succeed if it did happen). Also, it is technically possi…
Do you honestly believe any of the SQLite authors or its millions of users (especially deep-pocketed companies like Apple) lose any sleep at night over this?
Re: Limbo: A complete rewrite of SQLite in Rust
#180Earlier quoted context omitted.
That was my take when LibSQL was announced. And it still is and would be my take if LibSQL remains C-coded. But a Rust-coded rewrite of SQLite3 or LibSQL is a different story. The SQLite3 business model is that SQLite3 is open source but the best test suite for it is proprietary, and they don't accept contributions to any of either. This incentivizes anyone who needs support and/or new features in SQLite3 to join the…
> The SQLite3 business model is that SQLite3 is open source but the best test suite for it is proprietary no. the business model is services, and a red phone to companies who use sqlite in production. like nokia back in the days when we had these little flip phones, or desk phones had a "rolodesk" built in, or many other embedded uses of a little lovely dependable data store. the services include porting to and "cert…
> > The SQLite3 business model is that SQLite3 is open source but the best test suite for it is proprietary
> no.
> the business model is services, and a red phone to companies who use sqlite in production. like nokia back in the days when we had these little flip phones, or desk phones had a "rolodesk" built in, or many other embedded uses of a little lovely dependable data store.
Members of the SQLite Consortium surely have this "red phone" you speak of. So in what way was my characterization of their business model wrong?