Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Candle: Torch Replacement in Rust

github.com

Post reply on HN
Print view View original

31–40 of 192 posts

Re: Candle: Torch Replacement in Rust

#31

Post by tomrod »

freilanzer wrote: post #27
tomrod wrote: post #2

Oh man. I feel like this is a really big deal. I've been wondering what might move data scientists to Rust. Outside working through rustlings and trying new projects in Rust, I wonder if having this available will be it.

> I've been wondering what might move data scientists to Rust. If they didn't migrate to Julia they certainly won't migrate to Rust.

As someone who has made that choice in this context, Rust is low-level but an increasing part of the stack, and Julia always had weird edge cases for me.

Re: Candle: Torch Replacement in Rust

#32

Post by lionkor »

jstx1 wrote: post #29

It's cool that this is coming from huggingface and it isn't just someone's hobby project. Rust is in a weird place where people are trying to push it in a bunch of places where it doesn't belong naturally (web dev, data science) but then when you complain that it's difficult to use you get told that you're using the wrong language for your use case... I really want to like Rust - I've read the Rust book, done the Rus…

This has been my experience, too. Somewhere in Rust, there is a beautiful, functional(ish), pragmatic and simple language, hiding in between all the crappy things.

So it's just like C++ ;) except it has pattern matching and const by default.

Re: Candle: Torch Replacement in Rust

#34

Post by throwawaymaths »

chongli wrote: post #24
jstx1 wrote: post #13

Earlier quoted context omitted.

There's a bit of general dev snobbery to assume that data scientists can't code, or don't want to learn to code, or are able to understand so many other things but not a compiled language or a static type system.

It's not about ability to code, willingness to learn, or ability to understand. It's about friction. When you're iterating quickly and you're thinking about complicated mathematics and ideas, a complicated type system just adds a ton of friction. This is especially annoying when you know what you're doing is correct but the type system is unsatisfied and demands proof.

the place in DL/ML where typing could be the most useful is to keep you from trying to multiply differently sized tensors together. So the irony is that the one place where typing WOULD help DL/ML, is the place where rust is weak in its typing system (dependent types and const generics)

Re: Candle: Torch Replacement in Rust

#35

Post by ianbutler »

anentropic wrote: post #28
brrrrrm wrote: post #22

I've come to the personal conclusion that the issue with compiled languages and machine learning is that the API surface is so big and the operations themselves are very self-contained. I find that I am not typically writing complex "language-level" logic that would be expressible in any type of language and would benefit from the performance of compilation. I am guessing APIs and quickly testing ideas in totally unr…

This seems to be explicitly targeted at deploying ML models rather than developing them

Which makes sense their serving framework has a big python component right now, but most of it is written in rust. Given that one of the lead devs for the serving framework is also working on this, I'd guess they're going to look to remove that python portion.

Re: Candle: Torch Replacement in Rust

#36

Post by codeflo »

brrrrrm wrote: post #22

I've come to the personal conclusion that the issue with compiled languages and machine learning is that the API surface is so big and the operations themselves are very self-contained. I find that I am not typically writing complex "language-level" logic that would be expressible in any type of language and would benefit from the performance of compilation. I am guessing APIs and quickly testing ideas in totally unr…

I wonder about this. Is ML just plugging together prebuilt blocks because that covers the entire solution space, or are we limiting the solution space to that because that’s the only thing you can realistically do in Python?

Re: Candle: Torch Replacement in Rust

#37

Post by 1ba9115454 »

srush wrote: post #25
ianpurton wrote: post #9

I think the first Rust ML framework to gain massive traction will be the one that supports quantisation out of the box. At the moment there's no Rust ML library that does what llama.cpp or ggml can do.

Nowhere near as neat as candle or ggml, but just released a 4-bit rust llama2 implementation with simd. Runs pretty fast. https://github.com/srush/llama2.rs/

That's really cool.

Re: Candle: Torch Replacement in Rust

#38

Post by galangalalgol »

anentropic wrote: post #28
brrrrrm wrote: post #22

I've come to the personal conclusion that the issue with compiled languages and machine learning is that the API surface is so big and the operations themselves are very self-contained. I find that I am not typically writing complex "language-level" logic that would be expressible in any type of language and would benefit from the performance of compilation. I am guessing APIs and quickly testing ideas in totally unr…

This seems to be explicitly targeted at deploying ML models rather than developing them

I'm a novice at ml, would the performance benefit here be large compared to wonnx or the onnx bindings?

Re: Candle: Torch Replacement in Rust

#40

Post by robertlagrant »

codeflo wrote: post #36
brrrrrm wrote: post #22

I've come to the personal conclusion that the issue with compiled languages and machine learning is that the API surface is so big and the operations themselves are very self-contained. I find that I am not typically writing complex "language-level" logic that would be expressible in any type of language and would benefit from the performance of compilation. I am guessing APIs and quickly testing ideas in totally unr…

I wonder about this. Is ML just plugging together prebuilt blocks because that covers the entire solution space, or are we limiting the solution space to that because that’s the only thing you can realistically do in Python?

My sense is it's more the latter, at least when exploring how to solve things, but it's hard to say. What is the only thing you can realistically do in Python?
Post reply on HN