Any plans to open source the kv store?
Rust – A hard decision pays off
21–30 of 386 posts
Re: Rust – A hard decision pays off
#22> ..complex runtime issues which were almost impossible to reproduce or isolate > That’s when internal murmurs about a complete rewrite started brewing… > We decided to move our entire codebase to Rust > there was still one minor problem - no one on the team knew Rust "We have runtime issues so we will fix them by a complete rewrite in a language that no one on the team knows." I would not call that a "hard decision"…
I think of rolling the dice as taking a chance without any knowledge of the chance for it paying of.
Rust fit the need they had pretty well (on paper anyway since they didn’t have experience with it), and the industry is showing it to be a good choice for solving these types of problems. That’s not rolling the dice, it’s taking a chance based on research.
Re: Rust – A hard decision pays off
#23I don't want to be dismissive but how much of this success belongs to the team of rock solid senior engineers with enough experience under their belt rather than rewrite in Rust?
You can give some credit to the fact they were writing it a second time certainly, but as far as we know at least a lot of the people were the same ones who were central to writing V1. So certainly there was some lift from already doing it once, but if they went from a lot of subtle bugs they struggled to track down to having far fewer of them, and those bugs tended to be the types of issue the rust compiler and/or linter normally point out to you, it will certainly make it easier to mitigate the issues.
Rust cannot make a bad programmer a good programmer, but the fact it points out several classes of mistakes (sometimes too aggressively, hopefully someday we get Polonia and other improvements to the borrow checker to remove some things that are hard only because of the current implementation of various checking systems) allows far more confidence. I don't care how good someone is, having an automated way to validate large scales of issues aren't there is a big deal, and unlike unit/integration tests you don't have to write these.
Re: Rust – A hard decision pays off
#24So I was one of the 3 hackers who wrote the LSM that predated Rocks at FB, and as a result I distinctly remember when Rocks overtook it in features by trading off performance, and again when it met and exceeded the performance while keeping the greater capability. RocksDB kicks ass and has for a long time, I’d recommend it to anyone in the market for that sort of thing. FAISS is also great, and has been for years. I…
> But as someone who likes Rust and wants to like it more: don’t lead with that. The original title appears to be “Inside the Pinecone”, but seeing as they submitted it themselves I’m guessing they wanted the uptick on the Rust. The article itself doesn’t discuss their transition to Rust until the end. So I wouldn’t say they led with it originally, but perhaps didn’t get traction on the original title.
> seeing as they submitted it themselves
I'm grateful to muizelaar but I don't know who that is, so we can't take credit for this one.
Re: Rust – A hard decision pays off
#25I don't want to be dismissive but how much of this success belongs to the team of rock solid senior engineers with enough experience under their belt rather than rewrite in Rust?
A team with no rust experience
This mirrors my experience: getting buggy software out is easier and faster in JS (or whatever you're familiar without compile guarantees), getting something out to production is easier with rust.
Sometimes you have enough senior developers and discipline to reproduce and catch obscure bugs (which happen more or less based on your software complexity) and you can fix your mess but that's not always feasible.
Re: Rust – A hard decision pays off
#26So I was one of the 3 hackers who wrote the LSM that predated Rocks at FB, and as a result I distinctly remember when Rocks overtook it in features by trading off performance, and again when it met and exceeded the performance while keeping the greater capability. RocksDB kicks ass and has for a long time, I’d recommend it to anyone in the market for that sort of thing. FAISS is also great, and has been for years. I…
> But as someone who likes Rust and wants to like it more: don’t lead with that. The original title appears to be “Inside the Pinecone”, but seeing as they submitted it themselves I’m guessing they wanted the uptick on the Rust. The article itself doesn’t discuss their transition to Rust until the end. So I wouldn’t say they led with it originally, but perhaps didn’t get traction on the original title.
I guess I’m rooting for Rust in the long game sense, which is a different set of imperatives than the hang the hashtag on peripheral stuff sense.
I use a lot of great software written in Rust, it’s demonstrably a good vehicle for great software.
But too many of its fans are advocates, this can start to seem like an agenda. Don’t take engineering advice from people with an agenda.
Re: Rust – A hard decision pays off
#27Earlier quoted context omitted.
I would let the same team of "rock solid senior engineers" be the judge of that. They seem to think it mattered significantly enough.
Or ... since they are now betting fully on Rust, they now want everybody else on Rust too. Because more people == a higher probability that if they run into issues with Rust someone will fix it.
That's a really cynical take on the matter...
Re: Rust – A hard decision pays off
#28Maybe it’s just me but it seems irresponsible to move your entire team to a programming language none of them knows. Rust is great, but modern C++ is great too.
>but modern C++ is great too.
No, modern C++ is still cobbled together from tools built in a different era. It doesn't even come with a package manager/build system.
Just on the fact that cargo/crates.io exist and it has modules - it would have to be really bad to make me go back to C++ if I ever need to write something at that level.
Re: Rust – A hard decision pays off
#29so how are graphs used for vector search?
Re: Rust – A hard decision pays off
#30Maybe it’s just me but it seems irresponsible to move your entire team to a programming language none of them knows. Rust is great, but modern C++ is great too.
Yes. This comes across as "we are so smart, this project wasn't hard enough to demand our full attention".
Coding in a manifestly immature language signals that you are not really serious about being used where long-term support across a variety of platforms may be important. The language is still more likely than not to fizzle. If it does, it makes your flagship project a niche player.