Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Candle: Torch Replacement in Rust

github.com

Post reply on HN
Print view View original

91–100 of 192 posts

Re: Candle: Torch Replacement in Rust

#91

Post by alex_lav »

tomrod wrote: post #2

Oh man. I feel like this is a really big deal. I've been wondering what might move data scientists to Rust. Outside working through rustlings and trying new projects in Rust, I wonder if having this available will be it.

> I've been wondering what might move data scientists to Rust.

The two things that keep Python as the incumbent in this space is A.) It gets out of the way, sometimes to a fault, such that you can kinda write it without knowing what you're doing. This is a feature for most people, not a bug.

And B.) All the math/science libraries that already exist for it.

You can solve B, but until you solve A, Python stays put I think.

Re: Candle: Torch Replacement in Rust

#92

Post by PartiallyTyped »

codeflo wrote: post #36
brrrrrm wrote: post #22

I've come to the personal conclusion that the issue with compiled languages and machine learning is that the API surface is so big and the operations themselves are very self-contained. I find that I am not typically writing complex "language-level" logic that would be expressible in any type of language and would benefit from the performance of compilation. I am guessing APIs and quickly testing ideas in totally unr…

I wonder about this. Is ML just plugging together prebuilt blocks because that covers the entire solution space, or are we limiting the solution space to that because that’s the only thing you can realistically do in Python?

With ANNs all you are doing is matmul, addition, log, exp, cosine, sine, and indexing.

I am saying “all” because all operations we are doing are reduced to a tree of just these operations.

It turns out that these are enough, and that’s good because floating point units are limited. You don’t want to waste transistors on operations that don’t provide a lot of value.

Re: Candle: Torch Replacement in Rust

#93

Post by yieldcrv »

jstx1 wrote: post #29

It's cool that this is coming from huggingface and it isn't just someone's hobby project. Rust is in a weird place where people are trying to push it in a bunch of places where it doesn't belong naturally (web dev, data science) but then when you complain that it's difficult to use you get told that you're using the wrong language for your use case... I really want to like Rust - I've read the Rust book, done the Rus…

Best I can see is that it helps incremental time that adds up over time

Like if your build server is backed up maybe the rust version of your language's compiler and interpreter will make your build server not be backed up because it does everything faster

But for everyone else its not that useful to be using the rust version of everything

Its just another one of those things where you see lucrative job offers requiring rust, so you make up reasons to have impactful-ish experiences in rust

Re: Candle: Torch Replacement in Rust

#95

Post by coldtea »

devjab wrote: post #65
wokwokwok wrote: post #41

Earlier quoted context omitted.

Yeah, but the goal of this isn’t “use this instead of Python to hack up a model” …it’s: > And simply removing Python from production workloads. Python can really add overhead in more complex workflows and the GIL is a notorious source of headaches. You know, those hugging face guys kindaaaaa do know what they’re talking about. Like, really. Python is a pain in the ass in production environments.

> Python is a pain in the ass in production environments. I really dislike blanket statements like that one. Because any programming language can be a pain in production environments. One of the disadvantages of Python is also one of it's advantages. Because it's much easier to end up with a terrible production environment when you use "chaos" languages like Python or JavaScript, the tooling to avoid the pain points…

>Because any programming language can be a pain in production environments.

Except one with memory safety and static builds. Like Rust can offer.

Re: Candle: Torch Replacement in Rust

#96

Post by tikhonj »

Qworg wrote: post #82
tikhonj wrote: post #64

Earlier quoted context omitted.

I've worked on a Haskell + Rust project that followed roughly the dynamic you're describing and was pretty happy with it. I wrote some simple tooling that let us share algebraic types between the two to manage our interfaces and it worked pretty well. In our case we needed to serialize/deserialize data between our components so the interfaces were at a bit of a distance, but, in principle, it would have been totally…

I made the call between Haskell and Rust for my fintech startup. I chose Rust when I realized how few Haskell programmers there were available to hire - at 20 engineers, we'd be one of the largest shops on the West Coast of the US AFAIK. That said, we were one of the largest Rust shops outside of the FAANGs, but Rust was straightforward to teach to C/C++ folks.

We actually had the opposite experience in hiring for our Haskell team: it was substantially easier to find legitimately strong engineers for Haskell than for other teams using Python and Scala. I figure it comes down to two things:

1. A lot more people want to do Haskell than there are Haskell jobs

2. Haskell is an easy way to signal (show not tell) that our team is willing to do things differently—especially important because our team was at Target, which did not have much of a reputation as a tech employer!

We didn't try hiring Rust programmers because the existing Haskell folks were more than happy to pick it up.

So making hiring easier is one of the reasons I'd want to use Haskell, counterintuitively!

Re: Candle: Torch Replacement in Rust

#97

Post by [deleted] »

tomrod wrote: post #4
spenczar5 wrote: post #3

Earlier quoted context omitted.

I agree. Two years ago, I tried writing data-intensive components for Python as rust extension modules. It was painful, tedious, and confusing. I came back to it for a second try a month ago, and was blown away. PyO3 and maturin ( especially maturin) are stunningly straightforward. I actually find maturin easier to use than “normal” Python packaging tools, to the extent such a thing even exists. I really hope we will…

You've convinced me. I've been waiting to see when Julia or Rust would be able to start encroaching on Python's production lockup. Time to deep dive!

[deleted]

Re: Candle: Torch Replacement in Rust

#98

Post by swlkr »

jahewson wrote: post #66
unshavedyak wrote: post #42

Earlier quoted context omitted.

It may just be for different people, not sure. I'm a Rust fanatic, i use it for everything, including web dev. We even have a shop full of similar Rust folks. Personally it's my most productive language, and this is after years in Python, NodeJS, and ~5 years Go. Which isn't to attempt to invalidate your experience. But i do think: > writing Rust is a slog where you have to jump through hoops and spend a lot more tim…

Rust for web dev instead of Go? I want to believe. What’s the killer app?

Dioxus liveview or if you don't want websockets, dioxus fullstack, you can use warp or salvo but axum is good.

Rust embed is also great, I cross compile everything into a single dynamically linked binary and scp to my vps, deployment is a breeze.

Re: Candle: Torch Replacement in Rust

#99

Post by PartiallyTyped »

AlchemistCamp wrote: post #7
tomrod wrote: post #2

Oh man. I feel like this is a really big deal. I've been wondering what might move data scientists to Rust. Outside working through rustlings and trying new projects in Rust, I wonder if having this available will be it.

> I've been wondering what might move data scientists to Rust. I think dynamic typing is the missing piece. Rust's safety guarantees are great and I like the language, but the type system is a barrier for learners, especially people who aren't full-time devs. Ease of experimentation and flexibility are key for this use case.

The type system is certainly not a barrier, it’s actually rather straightforward unless I am missing something.

If you refer to lifetimes and borrow checker, those are different.

Re: Candle: Torch Replacement in Rust

#100

Post by MrJohz »

devjab wrote: post #65
wokwokwok wrote: post #41

Earlier quoted context omitted.

Yeah, but the goal of this isn’t “use this instead of Python to hack up a model” …it’s: > And simply removing Python from production workloads. Python can really add overhead in more complex workflows and the GIL is a notorious source of headaches. You know, those hugging face guys kindaaaaa do know what they’re talking about. Like, really. Python is a pain in the ass in production environments.

> Python is a pain in the ass in production environments. I really dislike blanket statements like that one. Because any programming language can be a pain in production environments. One of the disadvantages of Python is also one of it's advantages. Because it's much easier to end up with a terrible production environment when you use "chaos" languages like Python or JavaScript, the tooling to avoid the pain points…

> Because it's much easier to end up with a terrible production environment when you use "chaos" languages like Python or JavaScript, the tooling to avoid the pain points is so mature.

Could you describe what you mean, as in what that tooling is? Having done some funky deployments in Python before, I've never been very impressed by the ecosystem and tooling.

Post reply on HN