Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Candle: Torch Replacement in Rust

github.com

Post reply on HN
Print view View original

121–130 of 192 posts

Re: Candle: Torch Replacement in Rust

#121

Post by ramesh31 »

jstx1 wrote: post #29

It's cool that this is coming from huggingface and it isn't just someone's hobby project. Rust is in a weird place where people are trying to push it in a bunch of places where it doesn't belong naturally (web dev, data science) but then when you complain that it's difficult to use you get told that you're using the wrong language for your use case... I really want to like Rust - I've read the Rust book, done the Rus…

>So I'm not in a rush to rewrite anything written in Python just yet.

I think it's becoming clear that Python is to ML what JS is to the Web. We got stuck with it through a series of historical coincidences, and no one is super happy about it. But it mostly does the job and has evolved to fit the bill.

I'm not a huge fan but it's definitely here to stay. Hopefully Rust can take off but I don't see any other viable replacement.

Re: Candle: Torch Replacement in Rust

#122

Post by nextaccountic »

jahewson wrote: post #66
unshavedyak wrote: post #42

Earlier quoted context omitted.

It may just be for different people, not sure. I'm a Rust fanatic, i use it for everything, including web dev. We even have a shop full of similar Rust folks. Personally it's my most productive language, and this is after years in Python, NodeJS, and ~5 years Go. Which isn't to attempt to invalidate your experience. But i do think: > writing Rust is a slog where you have to jump through hoops and spend a lot more tim…

Rust for web dev instead of Go? I want to believe. What’s the killer app?

On the backend it's probably axum and tokio

https://github.com/tokio-rs/axum https://tokio.rs/tokio/tutorial

Re: Candle: Torch Replacement in Rust

#123

Post by bombela »

brundolf wrote: post #105
jstx1 wrote: post #29

It's cool that this is coming from huggingface and it isn't just someone's hobby project. Rust is in a weird place where people are trying to push it in a bunch of places where it doesn't belong naturally (web dev, data science) but then when you complain that it's difficult to use you get told that you're using the wrong language for your use case... I really want to like Rust - I've read the Rust book, done the Rus…

It'll never not have a trade-off! But there are some things that can make it easier in many cases: - Copy structs- ownership issues vanish when things can be bitwise-copied - .clone()- don't be afraid to clone outside of hot loops! If allocations are a problem, look at using Rc instead of Box where applicable - Pure functions/methods- you don't need mutable references if you aren't mutating things - Macros- reduce bo…

And the implementation behind those abstractions tend to be of really good quality. Thanks to the borrow checker and the scrutiny around the unsafe keyword.

In other words I am more willing to trust abstractions over tricky problems in Rust than I would in other languages.

Just for a second, try to imagine a generic memoize in C++.

Re: Candle: Torch Replacement in Rust

#124

Post by FL33TW00D »

ianpurton wrote: post #9

I think the first Rust ML framework to gain massive traction will be the one that supports quantisation out of the box. At the moment there's no Rust ML library that does what llama.cpp or ggml can do.

They already support quantization out of the box!

https://github.com/huggingface/candle/pull/314/files

Re: Candle: Torch Replacement in Rust

#126

Post by zengid »

jstx1 wrote: post #29

It's cool that this is coming from huggingface and it isn't just someone's hobby project. Rust is in a weird place where people are trying to push it in a bunch of places where it doesn't belong naturally (web dev, data science) but then when you complain that it's difficult to use you get told that you're using the wrong language for your use case... I really want to like Rust - I've read the Rust book, done the Rus…

I can attest to this. I really think it depends on your background and what your project is. Rust has a lot of baggage from it's use in systems programming that many folks coming from C++ already know, and a lot of strange functional programming things that ML-family language folks already know, so there's just a lot of new things crammed together. Now add move-by-default semantics, and tight restrictions on pointers, and it can get really hard. They key is getting to the point where you just try to use value semantics as much as possible, and stop trying to write everything perfectly the first time (i.e. use lots of `.clone` and `.unwrap` so you can get your code written down to do the thing you're trying to do).

I am hopeful that languages like Mojo and Val are going to find cleaner ways of doing what Rust does best, and that Rust can learn from them.

Re: Candle: Torch Replacement in Rust

#127

Post by sanderjd »

brrrrrm wrote: post #44
codeflo wrote: post #36

Earlier quoted context omitted.

I wonder about this. Is ML just plugging together prebuilt blocks because that covers the entire solution space, or are we limiting the solution space to that because that’s the only thing you can realistically do in Python?

It's worth noting that these pre-built blocks are both convenient and incredibly flexible. Before I started getting my hands dirty, it was hard to really see this. For example, here's a masked gather operation along the innermost dimension of an arbitrary rank tensor A: A[..., idx[mask]] I'd hate to have to express that in serial code and then pray a compiler could lift the correct vectorized version out of it.

Sure, but any language can provide that kind of pre-built block in a library. (Indeed, I suspect the library we're discussing has this operation, though I didn't check.) But if you want to implement a new operation that performs well, not every language can do that. In python, people tend to drop down to C or C++ or increasingly Rust to do that. But it seems (to me) like it would be nice to be able to use a single language for both things.

Re: Candle: Torch Replacement in Rust

#128

Post by [deleted] »

jstx1 wrote: post #29

It's cool that this is coming from huggingface and it isn't just someone's hobby project. Rust is in a weird place where people are trying to push it in a bunch of places where it doesn't belong naturally (web dev, data science) but then when you complain that it's difficult to use you get told that you're using the wrong language for your use case... I really want to like Rust - I've read the Rust book, done the Rus…

[deleted]

Re: Candle: Torch Replacement in Rust

#129

Post by pc2g4d »

What's the multithreading story? tch-rs's (and libtorch's) lack of real threading support was causing problems for my async code - at least, I haven't found a way to work around it yet, given my level of rust knowledge.

Tangentially: I keep thinking an ML framework could be done based on Rust macros, so the automatic differentiation is done entirely at compile time. `let grad = autodiff!(x + y);` (The `autodiff` crate does this at runtime IIUC). Then the compiler's optimizations can apply to the static computation graph.

I always thought that was the vision for TF in the early days, but clearly it moved in a more dynamic direction under PyTorch's influence.

Re: Candle: Torch Replacement in Rust

#130

Post by canadianfella »

bombela wrote: post #120
devjab wrote: post #65

Earlier quoted context omitted.

> Python is a pain in the ass in production environments. I really dislike blanket statements like that one. Because any programming language can be a pain in production environments. One of the disadvantages of Python is also one of it's advantages. Because it's much easier to end up with a terrible production environment when you use "chaos" languages like Python or JavaScript, the tooling to avoid the pain points…

My experience is that python is a mess in production. Even with a full CI, tests, mypy for static typing, pinned dependencies recursively, bazel with custom rules to build those dependencies (because many python dep have undeclared deps). And of course abysmal performances. And I wouldn't dare letting juniors touch python. Rust is far from perfect. But in my opinion formed by experience, Rust is the best imperative l…

[dead]
Post reply on HN