Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rewriting a high performance vector database in Rust

pinecone.io

Post reply on HN
Print view View original

131–140 of 157 posts

Re: Rewriting a high performance vector database in Rust

#131

Post by maccard »

krona wrote: post #61
pornel wrote: post #51

Earlier quoted context omitted.

Despite Rust's steep learning curve, it's also paradoxically easy to add novice Rust programmers to a project. This is because inexperienced Rust programmers are relatively harmless. Noob mistakes won't compile, rather than running into dangerous gotchas. You can tell noobs not to use `unsafe` (and there are ways to enforce that), and mostly they'll just write inefficient or non-idiomatic code, but the code will be f…

Memory sanitizers, address sanitizers, leak sanitizers, threading sanitizers, undefined behaviour sanitizers. The visual studio core guidelines checker. The clang-tidy core guideline checker. I could go on but my point is, the landscape does not really look like how you've painted it.

Assuming you have a CI suite dedicated to haning all of the above, sure. Meanwhile some of us are staring down the barrel of 30-60 minute compile times for a single configuration. Multiplying that out to compile with ubsan, asan, and running the static analysis tools over it would take probably 5-6x longer, _and_ we likely need to do it all twice to ensure were checking our "shipping" code paths.

It's not feasible to have a developer compile for 3 different platforms in two configurations with and without sanitizers, and core guidelines checkers for every change, and these tools take so long it's a huge cost.

Re: Rewriting a high performance vector database in Rust

#132

Post by tialaramex »

krona wrote: post #91
pornel wrote: post #72

Earlier quoted context omitted.

I know about these, but there is a marked difference between Rust and these tools. Static analysis tools have much harder job analyzing C++ (aliasing and escape analysis are way harder, and static analysis of thread-safety is basically impossible due to lack of thread-safety info in the type system). The results are a trade-off between being sparse or having false positives. The sanitizers only catch issues they can…

The results are a trade-off between sparse or having false positives. Rust just takes the other side of the trade-off, and will reject valid programs. Hence why the unsafe keyword exists, and why tools like Miri ( https://github.com/rust-lang/miri ) exist specifically for rust.

Sure, that's the consequence of Rice's theorem. "Is this program correct?" is formally Undecidable, to dodge that we split the programs three ways: Correct, Not Correct, shrug emoji.

It's obvious that Correct programs compile, Not Correct programs result in a compiler error, but what do we do with shrug emoji ? Rust says those go in the "Not Correct" pile and get a compiler error. C++ says - really, I'm not making this up - that they go in the Correct pile.

There's an immediate short term consequence, but also, after decades, a long term consequence that's arguably worse. Short term, C++ programmers can't know if their non-trivial program is Correct. It might be complete horse shit, the compiler won't necessarily tell them.

Long term, C++ grows worse and worse because there is no reason to shrink that shrug emoji category. All the programs in that category act as though they're fine, so there's no pressure whatsoever to improve the language standard, the compilers, etc.

Re: Rewriting a high performance vector database in Rust

#133

Post by anyfoo »

krona wrote: post #91
pornel wrote: post #72

Earlier quoted context omitted.

I know about these, but there is a marked difference between Rust and these tools. Static analysis tools have much harder job analyzing C++ (aliasing and escape analysis are way harder, and static analysis of thread-safety is basically impossible due to lack of thread-safety info in the type system). The results are a trade-off between being sparse or having false positives. The sanitizers only catch issues they can…

The results are a trade-off between sparse or having false positives. Rust just takes the other side of the trade-off, and will reject valid programs. Hence why the unsafe keyword exists, and why tools like Miri ( https://github.com/rust-lang/miri ) exist specifically for rust.

Over the decades, C started to reject more and more "valid" programs, and you have to use explicit casts to go from one pointer type to the other, or from an integer to a pointer. For obviously good reason.

Re: Rewriting a high performance vector database in Rust

#134

Post by estebank »

pjmlp wrote: post #95
mistrial9 wrote: post #28

Earlier quoted context omitted.

it is arguable that C++ in the modern days is no longer "one language" due to style, libraries, language features and code-base legacy; you have to find a coder that will fit your C++ world, not just C++

Just like it will happen to Rust when it achieves 30 years of history, getting features every six weeks. How many epochs will exist in 30 years?

As a rough estimate? 10, but I don't know what you think that means.

If you look at what [changes] were introduced by the [2018] and [2021] editions, they weren't as earth shattering as some might think:

2018:

- Module system changes

- Mandatory associated fn argument names

- dyn, async, await and try are now keywords

- You can't write `let s = libc::getenv(k.as_ptr()) as const _;` anymore, instead needing `let s = libc::getenv(k.as_ptr()) as const libc::c_char;` (Method dispatch for raw pointers to inference variables)

2021:

- TryInto, TryFrom and IntoIterator added to the prelude

- cargo dependency resolver changes

- [1, 2, 3].into_iterator() now works

- `|| a.x + 1` now captures `a.x`, not `a`

- Small technically backwards incompatible change to the panic macro formatting string

- any_identifier#, any_identifier"...", and any_identifier'...' are now reserved syntax

- Some previously existing warnings are now errors

- a | b is now matched in pat macro rules

[changes]: https://doc.rust-lang.org/edition-guide

[2018]: https://doc.rust-lang.org/edition-guide/rust-2018/index.html

[2021]: https://doc.rust-lang.org/edition-guide/rust-2021/index.html

Edit: it just dawned on me that by "epoch" you might not have meant the edition mechanism, which was at some point referred to as epochs and have so far happened every three years, but rather "how many iterations of idiomatic Rust code will there be in 30 years". If that was the original intent, you would also consider things like the introduction of the ? operator, or the upcoming let pat = expr else {}, or match ergonomics, or the likely deref patterns, or any number of features that on isolation might not be huge, but that can materially impact what idiomatic code looks like. I personally believe that that kind of iteration and evolution of a language is good and necessary. As long as forwards compatibility is maintained, and that forward compat doesn't hinder the future design space, making things better over time is a great thing!

Re: Rewriting a high performance vector database in Rust

#135

Post by tialaramex »

arriu wrote: post #47
kaladin_1 wrote: post #41

I have no problem with people rewriting their projects in whatever language they see fit. What stood out for me in the article is him saying that it's difficult getting developers with experience in both Python and C++. So, I wonder, if his in-house devs could pick up Rust that they previously couldn't write, why does he think he can not hire a good programmer and charge him to learn the stack the company uses. Why m…

I agree, rust has a difficult learning curve. I’ve often heard at least a year is required to really feel confident.

People really vary, it felt natural to me almost immediately but Rust's surveys indicate that some noticeable proportion of respondents report not being comfortable after more than a year.

This is seen as an important thing to improve because of the slogan "A language empowering everyone to build reliable and efficient software". If Rust isn't empowering everyone because say, 10% of people who try to pick it up can't get anywhere that's not good enough.

Re: Rewriting a high performance vector database in Rust

#136

Post by LarsDu88 »

kaladin_1 wrote: post #41

I have no problem with people rewriting their projects in whatever language they see fit. What stood out for me in the article is him saying that it's difficult getting developers with experience in both Python and C++. So, I wonder, if his in-house devs could pick up Rust that they previously couldn't write, why does he think he can not hire a good programmer and charge him to learn the stack the company uses. Why m…

I'm 3 weeks into Rust, and I've found it easy to get over an initial hump. I think where it becomes difficult is in thinking about smart pointers like Rc, RefCell, Arc, etc -- stuff I haven't even encountered yet. For things like certain popular leetcode questions, these concepts become more valuable.

Re: Rewriting a high performance vector database in Rust

#137

Post by saghm »

krona wrote: post #91
pornel wrote: post #72

Earlier quoted context omitted.

I know about these, but there is a marked difference between Rust and these tools. Static analysis tools have much harder job analyzing C++ (aliasing and escape analysis are way harder, and static analysis of thread-safety is basically impossible due to lack of thread-safety info in the type system). The results are a trade-off between being sparse or having false positives. The sanitizers only catch issues they can…

The results are a trade-off between sparse or having false positives. Rust just takes the other side of the trade-off, and will reject valid programs. Hence why the unsafe keyword exists, and why tools like Miri ( https://github.com/rust-lang/miri ) exist specifically for rust.

> Rust just takes the other side of the trade-off, and will reject valid programs

This has always been such a weird claim to me because it's not clear to me what's meant by "valid". My instinct is that it would be hard to define valid in a way without making most languages either accept some invalid programs or reject some valid ones (the exception being defining "valid" as "anything that language X accepts", but that wouldn't really say anything interesting about Rust). Sure, you could draw the line so that Rust is the only one that rejects valid programs, but is it worse to be the only language that rejects some valid programs than to be a language that accepts invalid ones? The alternative is that all languages accept some valid programs, at which point there's nothing specific about Rust that's worse until you start specifying which valid programs each language rejects. If there's a way to define "valid" so that Rust is the only one that rejects valid programs but other languages accept all valid and reject all invalid programs, I think it's non-obvious enough that it should be explicitly stated.

Re: Rewriting a high performance vector database in Rust

#138

Post by julienmarie »

smitty1e wrote: post #106

What is a vector database? https://www.pinecone.io/learn/vector-database/ ...was less than informative.

It's a database storing machine learning embeddings.

Example:

Let's suppose I've downloaded all the Gutemberg library books.

I can feed a transformer like Bert or GPT-3 to calculate the embeddings of these books.

These embeddings will represent in vector form (an array of numbers of fixed size) the meaning of these books.

I can save these vectors in this database and this database can then calculate the distance between these vectors, so basically how closely related they are in terms of topic.

If I query this database with the embedding of a sentence like "Love story between teenagers from 2 enemy families in Italy, they die at the end", hopefully the best result will be Romeo & Juliette.

I'm no ML person so take my comment with a grain of salt in terms of how well it works. But in theory, that's the goal.

Re: Rewriting a high performance vector database in Rust

#139

Post by za3faran »

pjmlp wrote: post #98
makmanalp wrote: post #45

> If you’re using a higher level language, you’re not going to have access to how the memory is laid out. A simple change, like removing indirection in our list, was an order of magnitude improvement in our latencies since there’s memory prefetching in the compiler and the CPU can anticipate which vectors are going to be loaded next in order to improve the memory footprint. This is a common experience and I'm still s…

Because it is a fake dichotomy. D, Nim, C#, Swift, not to count all of those that existed since Xerox PARC days.

And Java hopefully soon :)

Re: Rewriting a high performance vector database in Rust

#140

Post by AtlasBarfed »

lupire wrote: post #86

I liked the part where they said Python is too slow because it's garbage collected, and didn't show any metrics, and then built a new solution and Rust and didn't show metics to compare to the original system. Makes me think the eng lead just wanted to do Rust, and made up a rationalization.

Show me metrics where Python can compete with Java/Go, then we can bother with some discussion about Python vs C++/Rust. Actually, show me where Python is faster than Javascript.

But yes, it is a given that C++/Rust is faster than Python unless there is some fundamental algorithmic foolishness done by the C++/Rust programmer. People with industry experience know why that is a given.

Post reply on HN