Love the shoutut to dfdx. Haven't tried that crate yet but type checked matrix shapes sounds pretty good
Candle: Torch Replacement in Rust
21–30 of 192 posts
Re: Candle: Torch Replacement in Rust
#22I've come to the personal conclusion that the issue with compiled languages and machine learning is that the API surface is so big and the operations themselves are very self-contained.
I find that I am not typically writing complex "language-level" logic that would be expressible in any type of language and would benefit from the performance of compilation. I am guessing APIs and quickly testing ideas in totally unrelated colab notebooks. My productivity goes up when I can quickly answer questions like "how does this transpose impact the shape of the output" or "how do I find the mean of this tensor along only these dimensions."
Compiled languages, and especially Rust, really require you to go in with a plan. I find most of the machine learning code I write is inspired by rapidly playing with ideas.
Re: Candle: Torch Replacement in Rust
#23Earlier quoted context omitted.
> Rust's safety guarantees are great and I like the language, but the type system is a barrier for learners, especially people who aren't full-time devs. This surprises me. Hasn't everyone working with data (even at an "enthusiast" level) had to understand data types in the context of databases?
A lot of science/engineering/ML/data analysis happens from hacking around with only a vague idea of what to expect. Appeasing a type checker simultaneously is an enormous blow to productivity. Obviously possible, but challenging to get into a flow state when you cannot quickly iterate on ideas.
I mean you can still hold data in hashmaps no?
Or even serde::Value(s)? Its not that everything has to be a type
Re: Candle: Torch Replacement in Rust
#24Earlier quoted context omitted.
> Rust's safety guarantees are great and I like the language, but the type system is a barrier for learners, especially people who aren't full-time devs. This surprises me. Hasn't everyone working with data (even at an "enthusiast" level) had to understand data types in the context of databases?
There's a bit of general dev snobbery to assume that data scientists can't code, or don't want to learn to code, or are able to understand so many other things but not a compiled language or a static type system.
It's not about ability to code, willingness to learn, or ability to understand. It's about friction. When you're iterating quickly and you're thinking about complicated mathematics and ideas, a complicated type system just adds a ton of friction. This is especially annoying when you know what you're doing is correct but the type system is unsatisfied and demands proof.
Re: Candle: Torch Replacement in Rust
#25I think the first Rust ML framework to gain massive traction will be the one that supports quantisation out of the box. At the moment there's no Rust ML library that does what llama.cpp or ggml can do.
Nowhere near as neat as candle or ggml, but just released a 4-bit rust llama2 implementation with simd. Runs pretty fast.
Re: Candle: Torch Replacement in Rust
#26Oh man. I feel like this is a really big deal. I've been wondering what might move data scientists to Rust. Outside working through rustlings and trying new projects in Rust, I wonder if having this available will be it.
I sense that most folks will end up using this via a Python wrapper, if at all, much like how HF has Transformers written in Rust but with a Python wrapper.
This library is targeting production, a much smaller subset of engineers. See the 'why candle' section: https://github.com/huggingface/candle#why-candle. So you could use PyTorch to build models and Candle to serve them.
Re: Candle: Torch Replacement in Rust
#27Oh man. I feel like this is a really big deal. I've been wondering what might move data scientists to Rust. Outside working through rustlings and trying new projects in Rust, I wonder if having this available will be it.
> I've been wondering what might move data scientists to Rust.
If they didn't migrate to Julia they certainly won't migrate to Rust.
Re: Candle: Torch Replacement in Rust
#28I've come to the personal conclusion that the issue with compiled languages and machine learning is that the API surface is so big and the operations themselves are very self-contained. I find that I am not typically writing complex "language-level" logic that would be expressible in any type of language and would benefit from the performance of compilation. I am guessing APIs and quickly testing ideas in totally unr…
This seems to be explicitly targeted at deploying ML models rather than developing them
Re: Candle: Torch Replacement in Rust
#29It's cool that this is coming from huggingface and it isn't just someone's hobby project.
Rust is in a weird place where people are trying to push it in a bunch of places where it doesn't belong naturally (web dev, data science) but then when you complain that it's difficult to use you get told that you're using the wrong language for your use case...
I really want to like Rust - I've read the Rust book, done the Rustling exercises, read Zero to Production in Rust and did the project there, and still it's not clicking - writing Rust is a slog where you have to jump through hoops and spend a lot more time thinking about programming language abstractions than thinking about the problem you're solving.
So I'm not in a rush to rewrite anything written in Python just yet.
Re: Candle: Torch Replacement in Rust
#30Neat. This is the library I’ve been waiting to see. Rust has some nice potential but big parts of the ecosystem are missing. Now let’s see if it actually gets traction.