Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Candle: Torch Replacement in Rust

github.com

Post reply on HN
Print view View original

151–160 of 192 posts

Re: Candle: Torch Replacement in Rust

#151

Post by fnord123 »

maximilianburke wrote: post #109
alex_lav wrote: post #88

Earlier quoted context omitted.

This comment seems abnormally toxic for what's being discussed. > Yeah, but the goal of this isn’t “use this instead of Python to hack up a model” This was never said. > You know, those hugging face guys kindaaaaa do know what they’re talking about. This was never challenged. > Python is a pain in the ass in production environments. This is "I don't like python", but phrased adversarially. Language flamewars suck. So…

> This is "I don't like python", but phrased adversarially. But it is an incredible pain in the ass in production environments. Environment drift is something that _always_ seems to happen, no matter how diligent people are about using venvs or dependency managers or w/e. One of favorites was when we did a dependency upgrade (something that required a change to Click) we had an upgrade of Black forced on us in order…

>we had an upgrade of Black forced on us in order to satisfy Poetry which then reformatted all of our Python code.

Unlikely. If you are using pre-commit to run black then the version being used lives outside your venv and hence not subject to dependencies you are using. Poetry should also be outside your venv.

Re: Candle: Torch Replacement in Rust

#152

Post by progrus »

jstx1 wrote: post #29

It's cool that this is coming from huggingface and it isn't just someone's hobby project. Rust is in a weird place where people are trying to push it in a bunch of places where it doesn't belong naturally (web dev, data science) but then when you complain that it's difficult to use you get told that you're using the wrong language for your use case... I really want to like Rust - I've read the Rust book, done the Rus…

I absolutely love the slog of writing Rust… but I would not inflict it on my coworkers for most business-type projects.

Re: Candle: Torch Replacement in Rust

#153

Post by jononor »

brigadier132 wrote: post #115
Fiahil wrote: post #58

Earlier quoted context omitted.

Prototype in Python, but scale in Rust. I do a lot of AI Engineering in Python, but production Data Engineering, and Feature Engineering workloads are done in parallel with Rust. Simple and Fast. You can even make custom-made python modules with a Rust core when Numpy matrices are not enough to crack your graph-traversing problem efficiently ! (Thanks PyO3 and Maturin)

Since you mentioned Python / Rust interop I wouldn't be surprised to see this have a Python binding made for it and code will likely be easily translated over when it needs to be productionized.

I would be surprised if it does not happen. If it is not provided officially then very likey someone else will make it. Python is the lingua franca of the ML domain, after all.

Re: Candle: Torch Replacement in Rust

#154

Post by dgacmu »

throwawaymaths wrote: post #34
chongli wrote: post #24

Earlier quoted context omitted.

It's not about ability to code, willingness to learn, or ability to understand. It's about friction. When you're iterating quickly and you're thinking about complicated mathematics and ideas, a complicated type system just adds a ton of friction. This is especially annoying when you know what you're doing is correct but the type system is unsatisfied and demands proof.

the place in DL/ML where typing could be the most useful is to keep you from trying to multiply differently sized tensors together. So the irony is that the one place where typing WOULD help DL/ML, is the place where rust is weak in its typing system (dependent types and const generics)

I keep checking the progress on dfdx for this reason. It does what I (and, I assume from context, you) want: Provides static checking of tensor shapes. Which is fantastic. Not quite as much inference as I'd like but I love getting compile-time errors that I forgot to transpose before a matmul.

It depends on the generic_const_exprs feature which is still, to quote, "highly experimental":

https://github.com/rust-lang/rust/issues/76560

Definitely not for production use, but it gives a flavor for where things can head in the medium term, and it's .. it's nice. You could imagine future type support allowing even more inference for some intermediate shapes, of course, but even what it has now is really nice. Like this cute little convnet example:

https://github.com/coreylowman/dfdx/blob/main/examples/night...

Re: Candle: Torch Replacement in Rust

#155

Post by l-m-z »

pc2g4d wrote: post #129

What's the multithreading story? tch-rs's (and libtorch's) lack of real threading support was causing problems for my async code - at least, I haven't found a way to work around it yet, given my level of rust knowledge. Tangentially: I keep thinking an ML framework could be done based on Rust macros, so the automatic differentiation is done entirely at compile time. `let grad = autodiff!(x + y);` (The `autodiff` crat…

The tensors should be Send and Sync so can be manipulated from multiple threads, the underlying data is protected by a RWLock to guard against data races. The heavy operations such as matrix multiplication will be run on multiple cores even without using some explicit threading.

Re: Candle: Torch Replacement in Rust

#156

Post by jononor »

alex_lav wrote: post #91
tomrod wrote: post #2

Oh man. I feel like this is a really big deal. I've been wondering what might move data scientists to Rust. Outside working through rustlings and trying new projects in Rust, I wonder if having this available will be it.

> I've been wondering what might move data scientists to Rust. The two things that keep Python as the incumbent in this space is A.) It gets out of the way, sometimes to a fault, such that you can kinda write it without knowing what you're doing. This is a feature for most people, not a bug. And B.) All the math/science libraries that already exist for it. You can solve B, but until you solve A, Python stays put I th…

And C) it is taught at every University that teaches ML/DS.

I think unseating Python as the lingua franca will takes as long as it has taken to unseat MATLAB in engineering. 20+ years...

Of course other things will also exist, and all languages will have decent ML support (especially for inference).

Re: Candle: Torch Replacement in Rust

#157

Post by mkehrt »

orangetuba wrote: post #142
unshavedyak wrote: post #42

Earlier quoted context omitted.

It may just be for different people, not sure. I'm a Rust fanatic, i use it for everything, including web dev. We even have a shop full of similar Rust folks. Personally it's my most productive language, and this is after years in Python, NodeJS, and ~5 years Go. Which isn't to attempt to invalidate your experience. But i do think: > writing Rust is a slog where you have to jump through hoops and spend a lot more tim…

So if you take a Python program and you add the mental effort of having to deal with static types, lifetimes, the borrow checker and performance considerations, and this added mental effort actually allows you to write the program faster? That is incredible. It's like being able to carve- and install a door from a slab of wood faster than installing a prefabricated door.

This line of reasoning is utterly alien to me. If your types add mental overhead, you’re doing something wrong. You should be able to rely on the typechecker to check invariants for you, decreasing mental overhead. You have to think about types anyway, especially in an untyped language, where you can’t rely on the typechecker.

Re: Candle: Torch Replacement in Rust

#158

Post by maximilianburke »

fnord123 wrote: post #151
maximilianburke wrote: post #109

Earlier quoted context omitted.

> This is "I don't like python", but phrased adversarially. But it is an incredible pain in the ass in production environments. Environment drift is something that _always_ seems to happen, no matter how diligent people are about using venvs or dependency managers or w/e. One of favorites was when we did a dependency upgrade (something that required a change to Click) we had an upgrade of Black forced on us in order…

>we had an upgrade of Black forced on us in order to satisfy Poetry which then reformatted all of our Python code. Unlikely. If you are using pre-commit to run black then the version being used lives outside your venv and hence not subject to dependencies you are using. Poetry should also be outside your venv.

It's not "unlikely"; it's something that happened.

And, again, if something that was configured wrong this goes back to how managing Python is a pain in the ass. I should be able to dedicate my brain power to the problems I need to solve, not what things need to be in my venv and what things need to be outside of it.

Re: Candle: Torch Replacement in Rust

#159

Post by mkehrt »

synergy20 wrote: post #46
jstx1 wrote: post #29

It's cool that this is coming from huggingface and it isn't just someone's hobby project. Rust is in a weird place where people are trying to push it in a bunch of places where it doesn't belong naturally (web dev, data science) but then when you complain that it's difficult to use you get told that you're using the wrong language for your use case... I really want to like Rust - I've read the Rust book, done the Rus…

A similar situation here for me. Rust might be a modern Ada for a particular use case(system programming), I was puzzled why stackoverflow ranked it as 'the most loved' language for years. For each language related post at HN, you will always see rust lover's posts there saying "I wish it's in rust", or "there is a rust version", or "rust is even better". The reality does not match up with the enthusiasm for me thoug…

I write rust professionally and I really, really enjoy it. It’s lots of fun, and has found a good (and novel!) point in the space of safety vs. speed. That said, I think in the 80-90% case, a language with a garbage collector is probably more appropriate for the task at hand.

Re: Candle: Torch Replacement in Rust

#160

Post by alex_lav »

maximilianburke wrote: post #109
alex_lav wrote: post #88

Earlier quoted context omitted.

This comment seems abnormally toxic for what's being discussed. > Yeah, but the goal of this isn’t “use this instead of Python to hack up a model” This was never said. > You know, those hugging face guys kindaaaaa do know what they’re talking about. This was never challenged. > Python is a pain in the ass in production environments. This is "I don't like python", but phrased adversarially. Language flamewars suck. So…

> This is "I don't like python", but phrased adversarially. But it is an incredible pain in the ass in production environments. Environment drift is something that _always_ seems to happen, no matter how diligent people are about using venvs or dependency managers or w/e. One of favorites was when we did a dependency upgrade (something that required a change to Click) we had an upgrade of Black forced on us in order…

> But it is an incredible pain in the ass in production environments.

"I find it hard to maintain Python in production environments", but worded adversarially.

> One of favorites was when we did a dependency upgrade (something that required a change to Click) we had an upgrade of Black forced on us in order to satisfy Poetry which then reformatted all of our Python code.

This is literally just a misconfigured set of tools then.

> There's always someone saying "you're using it wrong" or "you're doing something worng", even if you follow the guides and howtos to get started.

This is true of all languages and frameworks. There are people _in this comment section_ insisting those that have had negative Rust experiences must simply be doing it wrong. This is more a comment on our industry than any individual technology.

> For a language which states "There should be one -- and preferably only one -- obvious way to do it"

Yes, this has always been a bold faced lie from the Python community. Marketing speak is marketing speak.

> that there is no single one way to use it right in production -- let alone an obvious one -- that makes it, simply, a "pain in the ass".

It's as much a pain in the ass as literally every piece of technology, as the above statement is true for every technology and framework that exists.

I acknowledge that you don't like it. That's great! But again, language flamewars suck, and suggesting that something is bad because you've individually had a bad experience is pretty silly. Lots of people successfully deploy and maintain python applications of various sizes without referring to the process as "a pain in the ass".

Post reply on HN