Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Candle: Torch Replacement in Rust

github.com

Post reply on HN
Print view View original

101–110 of 192 posts

Re: Candle: Torch Replacement in Rust

#101

Post by nmstoker »

alex_lav wrote: post #88
wokwokwok wrote: post #41

Earlier quoted context omitted.

Yeah, but the goal of this isn’t “use this instead of Python to hack up a model” …it’s: > And simply removing Python from production workloads. Python can really add overhead in more complex workflows and the GIL is a notorious source of headaches. You know, those hugging face guys kindaaaaa do know what they’re talking about. Like, really. Python is a pain in the ass in production environments.

This comment seems abnormally toxic for what's being discussed. > Yeah, but the goal of this isn’t “use this instead of Python to hack up a model” This was never said. > You know, those hugging face guys kindaaaaa do know what they’re talking about. This was never challenged. > Python is a pain in the ass in production environments. This is "I don't like python", but phrased adversarially. Language flamewars suck. So…

I'm not getting a sense of toxicity from this discussion. Sure, people are disagreeing and have distinct positions but the manner of discussion seems constructive.

>> This was never said.

Quotes don't have to used solely to indicate somebody explicitly said something.

Re: Candle: Torch Replacement in Rust

#103

Post by GaggiX »

liuliu wrote: post #81
GaggiX wrote: post #75

Earlier quoted context omitted.

I cannot really see how it can be a worse experience than waiting for the model to run, fix the error found, wait a second time, fix another error; when you can just check instantly if the model architecture is sound.

Because that's not my workflow looks like? * Fire up Jupyter notebook, create some PyTorch nn.Module, validate the output is expected. * Built out the model, validate the output is expected. * Check the data shape, do a eval() run with the data. * Move out of Jupyter notebook (or not), to actually run the trainer. Each of the "build out model" phase, shape inference at compile time would be kind of annoying especiall…

That's also my workflow but the "Check the data shape, do a eval() run with the data." is the annoying process I was describing, the iterative "waiting for the model to run", "see that it doesn't work", "print random tensor shape trying the debug the code" etc... There is nothing annoying about having an oracle that can check it instantly unless you think that suffering should be part of the experience.

Re: Candle: Torch Replacement in Rust

#104

Post by Animats »

Oh, another Candle and Torch. There are also programs called "Candle" and "Torch" inside the WIX toolset, a widely used generator of Microsoft Windows installers. (Not to be confused with Wix.com, the heavily promoted low-end web hosting service.)

Re: Candle: Torch Replacement in Rust

#105

Post by brundolf »

jstx1 wrote: post #29

It's cool that this is coming from huggingface and it isn't just someone's hobby project. Rust is in a weird place where people are trying to push it in a bunch of places where it doesn't belong naturally (web dev, data science) but then when you complain that it's difficult to use you get told that you're using the wrong language for your use case... I really want to like Rust - I've read the Rust book, done the Rus…

It'll never not have a trade-off! But there are some things that can make it easier in many cases:

- Copy structs- ownership issues vanish when things can be bitwise-copied

- .clone()- don't be afraid to clone outside of hot loops! If allocations are a problem, look at using Rc instead of Box where applicable

- Pure functions/methods- you don't need mutable references if you aren't mutating things

- Macros- reduce boilerplate

- Purpose-built traits- you can hang methods off of different structs, including std or builtin structs, that can make certain flows really ergonomic

In general: Rust asks you to care about all the little details at a language level, but it also empowers you in different ways to build high-level abstractions on top of those details so that you often don't have to care about them. I'm finding that I have the smoothest time with it when I use traits and macros to make my own little DSL for the problem I'm trying to solve, and then write my business logic in that.

And then- many libraries offer you their own abstractions that give you the same sort of benefit. Concurrency normally involves a bunch of ownership and locking headaches, but something like rayon makes it breezy by giving you an abstraction that handles those details for you. Async web route handlers with shared state from scratch would normally be hard, but axum and Rocket give you abstractions that handle the gross stuff for you. One of my favorite crates I recently discovered is just called "memoize", and it just gives you a macro you can slap on any old function to memoize its results! (global mutable state!) https://crates.io/crates/memoize. This would be a mess to implement yourself, but the abstraction makes it breezy

And then personally: the main reason I'd use a Rust library like the OP over a Python library is the tooling/dependency management. Running a Rust project with dependencies Just Works and that's extremely valuable to me. But, it'll always be weighed against the costs of using a lower-level language.

Re: Candle: Torch Replacement in Rust

#107

Post by brigadier132 »

jstx1 wrote: post #29

It's cool that this is coming from huggingface and it isn't just someone's hobby project. Rust is in a weird place where people are trying to push it in a bunch of places where it doesn't belong naturally (web dev, data science) but then when you complain that it's difficult to use you get told that you're using the wrong language for your use case... I really want to like Rust - I've read the Rust book, done the Rus…

There are two kinds of complexity. Upfront complexity and hidden complexity. Rust with the borrow checker significantly shrinks the space of possible programs that can be written to solve a problem. Some people think this is bad because they can't write software in a way they find natural, I like it because of the possible programs that rust makes impossible to write the vast majority are incorrect and have bugs.

I also really like that when I use libraries, I know that all the library authors also subject to these same restrictions and I can trust their code much more than I could in other languages.

Re: Candle: Torch Replacement in Rust

#109

Post by maximilianburke »

alex_lav wrote: post #88
wokwokwok wrote: post #41

Earlier quoted context omitted.

Yeah, but the goal of this isn’t “use this instead of Python to hack up a model” …it’s: > And simply removing Python from production workloads. Python can really add overhead in more complex workflows and the GIL is a notorious source of headaches. You know, those hugging face guys kindaaaaa do know what they’re talking about. Like, really. Python is a pain in the ass in production environments.

This comment seems abnormally toxic for what's being discussed. > Yeah, but the goal of this isn’t “use this instead of Python to hack up a model” This was never said. > You know, those hugging face guys kindaaaaa do know what they’re talking about. This was never challenged. > Python is a pain in the ass in production environments. This is "I don't like python", but phrased adversarially. Language flamewars suck. So…

> This is "I don't like python", but phrased adversarially.

But it is an incredible pain in the ass in production environments.

Environment drift is something that _always_ seems to happen, no matter how diligent people are about using venvs or dependency managers or w/e.

One of favorites was when we did a dependency upgrade (something that required a change to Click) we had an upgrade of Black forced on us in order to satisfy Poetry which then reformatted all of our Python code.

Recently we had mypy stop checking one of our internal packages because of a Dockerfile refactoring. The package could still be found at runtime but the type information just wasn't found by mypy. We didn't notice it until we started getting errors at runtime, which made the fix a lot more difficult to diagnose.

There's always someone saying "you're using it wrong" or "you're doing something worng", even if you follow the guides and howtos to get started.

For a language which states "There should be one -- and preferably only one -- obvious way to do it" as part of its guiding principles, that there is no single one way to use it right in production -- let alone an obvious one -- that makes it, simply, a "pain in the ass".

Re: Candle: Torch Replacement in Rust

#110

Post by rizky05 »

tracker1 wrote: post #76
jahewson wrote: post #66

Earlier quoted context omitted.

Rust for web dev instead of Go? I want to believe. What’s the killer app?

I would say for me, the killer app would be just about anything you might reach for Electron with, but want a much smaller install footprint. Tauri in and of itself is pretty cool, kind of a lighter alternative to electron using native platform browser engine. In general, I would probably reach for Rust if I needed to do anything wither Web-Assembly (wasm) as the path to working is much nicer than other languages imo…

[dead]
Post reply on HN