Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Limbo: A complete rewrite of SQLite in Rust

turso.tech

Post reply on HN
Print view View original

141–150 of 238 posts

Re: Limbo: A complete rewrite of SQLite in Rust

#141

Post by throwawaymaths »

wis wrote: post #61
masklinn wrote: post #41

Earlier quoted context omitted.

Seems like a potentially interesting project to get rid of sqlite's compatibility baggage e.g. non-strict tables, opt-in foreign keys, the oddities around rowid tables, etc... as well as progress the dialect a bit (types and domains for instance).

But the article mentions that they intend to have full compatibility: > Our goal is to build a reimplementation of SQLite from scratch, fully compatible at the language and file format level, with the same or higher reliability SQLite is known for, but with full memory safety and on a new, modern architecture.

i would guess "full memory safety" is going to be impossible, at least at compile time. I'd guess that if for no other reason than performance SQLite uses data oriented techniques that effectively reduces pointers to indices, which will no longer have ownership or lifetime tracking in the rust compiler.

Re: Limbo: A complete rewrite of SQLite in Rust

#142

Post by glommer »

wis wrote: post #61
masklinn wrote: post #41

Earlier quoted context omitted.

Seems like a potentially interesting project to get rid of sqlite's compatibility baggage e.g. non-strict tables, opt-in foreign keys, the oddities around rowid tables, etc... as well as progress the dialect a bit (types and domains for instance).

But the article mentions that they intend to have full compatibility: > Our goal is to build a reimplementation of SQLite from scratch, fully compatible at the language and file format level, with the same or higher reliability SQLite is known for, but with full memory safety and on a new, modern architecture.

author here: fully compatible at the language and file format level.

Further down the post I actually call out explicitly that we do intend to get rid of some of the baggage.

Re: Limbo: A complete rewrite of SQLite in Rust

#143

Post by glommer »

_ugfj wrote: post #63
jey wrote: post #35

Earlier quoted context omitted.

That's fully in line with what they're announcing here. It's the announcement of a new project that has passed the prototyping stage, but one that has not reached the 1.0 stage.

And it never will The first 90% is easy, it's the second 90% that is very hard.

dunno, we rewrote a database before, much larger and harder than sqlite, and it was pretty successful.

In my experience the 10% that doesn't get done is the 10% that people don't care too much about it anyway.

Re: Limbo: A complete rewrite of SQLite in Rust

#145

Post by glommer »

pkolaczk wrote: post #64
glommer wrote: post #48

Earlier quoted context omitted.

The goal here is not to claim that it is faster, though (it isn't, in a lot of other things it is slower and if you run cargo bench you will see) It is to highlight that we already reached a good level of performance this early in the project. Your claim about the programming language having no impact is just false, though. It's exactly what people said back in 2015 when we released Scylla. It was already false then,…

> It is to highlight that we already reached a good level of performance this early in the project. This is the right thing to do. It's a pity so many projects don't keep an eye on performance from the very first day. Getting high performing product is a process, not a single task you apply at the end. Especially in a performance critical system like a database, if you don't pay attention to performance and instead y…

thanks. I am sad, but not that surprised, that a lot of people here are interpreting this as we claiming that we're already faster than sqlite all over.

I don't even care about being faster than sqlite, just not being slower, this early, is already the win I'm looking for.

Re: Limbo: A complete rewrite of SQLite in Rust

#146

Post by akira2501 »

cryptonector wrote: post #21
danieljanes wrote: post #12

Given the code quality and rigid testing, SQLite is probably the last project that should be rewritten. It'd be great to see all other C code rewritten first!

That was my take when LibSQL was announced. And it still is and would be my take if LibSQL remains C-coded. But a Rust-coded rewrite of SQLite3 or LibSQL is a different story. The SQLite3 business model is that SQLite3 is open source but the best test suite for it is proprietary, and they don't accept contributions to any of either. This incentivizes anyone who needs support and/or new features in SQLite3 to join the…

> The U.S. government wants everyone to abandon C/C++

That's the position of two federal agencies, namely, FBI and CISA. They don't describe how this change will reduce CVEs or why the languages they prefer still produce projects with CVEs.

I don't particularly hold the technical or social acumen of FBI or CISA in particularly high regard and I'm not sure why anyone would by default either. Mostly because they say things like "switch to python!" without once accounting for the fact that python is written in C.

It's an absurd point to invoke as a defense of this idea.

Re: Limbo: A complete rewrite of SQLite in Rust

#147

Post by akira2501 »

kabirgoel wrote: post #3
toenail wrote: post #2

Dunno. Good luck to them, but I never saw a need to rewrite sqlite.

Guessing the shortcomings become starker if you’re spending lots of time in the codebase/building a company on top of it.

> building a company on top of it.

So be sure you proceed in such a way that never contributes any money or code back to the original project.

Re: Limbo: A complete rewrite of SQLite in Rust

#148

Post by cy7 »

wongarsu wrote: post #98
bruce511 wrote: post #59

Earlier quoted context omitted.

>> The SQLite3 business model is that SQLite3 is open source This is going to sound pedantic, but SQLite is not Open Source. It's Public Domain. The distinction is subtle, but it is important.

It's a difference only insofar that in many jurisdictions their claim that it's public domain has no legal value. If it was truly public domain (e.g. if the authors were long dead) it would be open source. But far from all places allow you to arbitrarily put things in the public domain. I'm a bit puzzled why SQLite doesn't solve this trivial issue by claiming the code is CC0-licensed. CC0 is made just for that: a ver…

> sell you a fancy legally-satisfying piece of paper for an undisclosed price

It's $6,000 https://sqlite.org/prosupport.html

Re: Limbo: A complete rewrite of SQLite in Rust

#149

Post by ncruces »

All this talk of “SQLite is not open contribution” never seems to consider that a project being “open contribution” doesn't mean the maintainers will accept your contributions.

They have a process for contributions to follow: you suggest a feature, they implement it. It's far from the only project to take such a stance.

Just in the SQLite “ecosystem” see the contribution policies of Litestream and LiteFS. I don't see people brandishing the ”not open contribution” to Ben's projects.

https://github.com/superfly/litefs?tab=readme-ov-file#contri...

https://github.com/benbjohnson/litestream?tab=readme-ov-file...

Re: Limbo: A complete rewrite of SQLite in Rust

#150

Post by AlotOfReading »

0x457 wrote: post #101
AlotOfReading wrote: post #27

Earlier quoted context omitted.

There's still an element of truth in the idea that C is going to be faster by default. There's simply a much lower bar to writing fast (and unsafe) C. Fast Rust demands considerably more thoughtfulness from the programmer (at least for me).

> Fast Rust demands considerably more thoughtfulness from the programmer (at least for me). While fast code requires thoughtfulness regardless of the language, I think rust lets you focus on the fast aspect more because rustc ensures _some_ safety and correctness. I can write fast and very unsafe C code fast, but I write code that just as fast , but safer in rust faster than in C.

It's entirely likely that you could write faster Rust in the same (nigh-infinite) time it'd take to write equally safe C. I intentionally avoided that comparison though. If you take a normal 10min function in C, it's going to compile into something reasonable and run fast. If you take the same 10min rust function, the language surface area is so much larger that there's a much higher chance that it won't.

Here's a more concrete, albeit irrelevant in practice example from writing most things in both languages:

https://news.ycombinator.com/item?id=42342382#42352053

Implemented in Rust over generic T, you need Wrapping or the equivalent num_traits traits. The implementations for these take borrowed references. Rustc is pretty good at ensuring this becomes pass by value under the hood, but it's imperfect. I found instances of it failing in the test disassembly, even though an implementation for this never has to touch anything but registers. That's performance work that wouldn't have existed in C/C++ for these particular types.

Post reply on HN