Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Candle: Torch Replacement in Rust

github.com

Post reply on HN
Print view View original

51–60 of 192 posts

Re: Candle: Torch Replacement in Rust

#51

Post by josephg »

spenczar5 wrote: post #12
CharlesW wrote: post #10

Earlier quoted context omitted.

> Rust's safety guarantees are great and I like the language, but the type system is a barrier for learners, especially people who aren't full-time devs. This surprises me. Hasn't everyone working with data (even at an "enthusiast" level) had to understand data types in the context of databases?

Yes, but Rust’s type system is waaaaay more sophisticated than your typical SQL database. Talk about “generic associated types” and you can see people start to shrink back in fear. Plus, a surprisingly enormous fraction of the NoSQL hysteria was that “you don’t even need a schema!”

> Plus, a surprisingly enormous fraction of the NoSQL hysteria was that “you don’t even need a schema!”

The problem with this is that you have a schema regardless. In "schemaless" systems, you still have a schema. You just don't have it written down anywhere. And because its not written down, the database doesn't enforce your schema. So its very common for nosql databases to have records with missing fields or subtly corrupted data.

That said, there's a reason why nosql became popular. I think a large part of it is that SQL databases are too inflexible. You have to define the entire schema ahead of time, and schema evolution is way too difficult. When I'm writing software, I don't define all of my columns ahead of time. I want to add and change stuff as I go.

I wish databases were more flexible. Like, let me just add arbitrary extra fields to my records. Let the database recommend promoting a dynamic field to a static field. ("Hey, all your records have a name: string field. Wanna make that required?").

I love programming languages with similar features. I love typescript for prototyping because I can start with the type any (or wildcard fields in my interfaces - like [k: string]: any). And then I can formalize my types later when I understand my software better.

I don't think schemas are a mistake for beginners. I think the opposite - they're amazing guard rails.

But I agree with other commenters here. Rust's types include information about ownership, lifetime, containment (Box / Rc), traits and the stored data type all in one go. Its terrifying to beginners and exhausting for experts to use. I've been writing rust for years, and the Future trait still scares me.

Re: Candle: Torch Replacement in Rust

#52

Post by ElectricalUnion »

DannyBee wrote: post #5
tomrod wrote: post #2

Oh man. I feel like this is a really big deal. I've been wondering what might move data scientists to Rust. Outside working through rustlings and trying new projects in Rust, I wonder if having this available will be it.

"I've been wondering what might move data scientists to Rust" I mean, writ large, the answer is "The availability of the ecosystem they need to use daily in Rust, without having to rewrite it themselves." That is more than just a torch replacement, so this won't do it for data scientists, even if Python is ... painful in a lot of ways to productionize or even use. Maybe it'll get some started on that ecosystem, thoug…

I guess the answer is when the ecosystem stops being composed of mostly experimentation, training and prototyping (data scientists) and turns into deployment and upkeep (ops) of said systems.

Prototyping and training doesn't pay the bills. (unless you're Hugging Face, lol)

Re: Candle: Torch Replacement in Rust

#54

Post by spoiler »

jstx1 wrote: post #29

It's cool that this is coming from huggingface and it isn't just someone's hobby project. Rust is in a weird place where people are trying to push it in a bunch of places where it doesn't belong naturally (web dev, data science) but then when you complain that it's difficult to use you get told that you're using the wrong language for your use case... I really want to like Rust - I've read the Rust book, done the Rus…

> writing Rust is a slog where you have to jump through hoops and spend a lot more time thinking about programming language abstractions than thinking about the problem you're solving.

Have you tried "sticking" with it for a while? Try forcing yoursel to use it[1]. I find that's the best way to learn it (and try reading more code from open source project in Rust you like).

I have an opposite experience writing Rust: it allows me to focus on the problem more easily, especially when the problem domain is complex and the codebase grows.

[1] a trick often used while learning human languages

Re: Candle: Torch Replacement in Rust

#55

Post by josephg »

delijati wrote: post #39

How big are the executabels ? I would like to move our interence code from tensorflow to a executable

I just compiled the llama example on my M1 mac. The resulting binary is 9.5mb in size. I assume that includes support for apple's ML acceleration but not CUDA.

Re: Candle: Torch Replacement in Rust

#56

Post by dkarl »

jstx1 wrote: post #29

It's cool that this is coming from huggingface and it isn't just someone's hobby project. Rust is in a weird place where people are trying to push it in a bunch of places where it doesn't belong naturally (web dev, data science) but then when you complain that it's difficult to use you get told that you're using the wrong language for your use case... I really want to like Rust - I've read the Rust book, done the Rus…

Rust needs a prototyping/scripting/application language on top of it, with 80% of the speed and 99% of the safety, with automatic memory management (reference counting or GC) and at least one idiomatic way of doing concurrency that disallows sharing mutable state. A language that is tailor-made for building applications quickly on top of the Rust ecosystem, with a build tool that gracefully handles mixed projects if you need to build part of your application in Rust, and support for writing small shebang scripts.

Also, I'd like a pony, a jetpack, and VIP passes to the premiere of Dune: Part 2.

Re: Candle: Torch Replacement in Rust

#57

Post by ravenstine »

wokwokwok wrote: post #41
jstx1 wrote: post #29

It's cool that this is coming from huggingface and it isn't just someone's hobby project. Rust is in a weird place where people are trying to push it in a bunch of places where it doesn't belong naturally (web dev, data science) but then when you complain that it's difficult to use you get told that you're using the wrong language for your use case... I really want to like Rust - I've read the Rust book, done the Rus…

Yeah, but the goal of this isn’t “use this instead of Python to hack up a model” …it’s: > And simply removing Python from production workloads. Python can really add overhead in more complex workflows and the GIL is a notorious source of headaches. You know, those hugging face guys kindaaaaa do know what they’re talking about. Like, really. Python is a pain in the ass in production environments.

> You know, those hugging face guys kindaaaaa do know what they’re talking about.

> Like, really.

Trust me, bro?

Re: Candle: Torch Replacement in Rust

#58

Post by Fiahil »

jstx1 wrote: post #29

It's cool that this is coming from huggingface and it isn't just someone's hobby project. Rust is in a weird place where people are trying to push it in a bunch of places where it doesn't belong naturally (web dev, data science) but then when you complain that it's difficult to use you get told that you're using the wrong language for your use case... I really want to like Rust - I've read the Rust book, done the Rus…

Prototype in Python, but scale in Rust.

I do a lot of AI Engineering in Python, but production Data Engineering, and Feature Engineering workloads are done in parallel with Rust. Simple and Fast.

You can even make custom-made python modules with a Rust core when Numpy matrices are not enough to crack your graph-traversing problem efficiently ! (Thanks PyO3 and Maturin)

Re: Candle: Torch Replacement in Rust

#59

Post by fooblaster »

wokwokwok wrote: post #41
jstx1 wrote: post #29

It's cool that this is coming from huggingface and it isn't just someone's hobby project. Rust is in a weird place where people are trying to push it in a bunch of places where it doesn't belong naturally (web dev, data science) but then when you complain that it's difficult to use you get told that you're using the wrong language for your use case... I really want to like Rust - I've read the Rust book, done the Rus…

Yeah, but the goal of this isn’t “use this instead of Python to hack up a model” …it’s: > And simply removing Python from production workloads. Python can really add overhead in more complex workflows and the GIL is a notorious source of headaches. You know, those hugging face guys kindaaaaa do know what they’re talking about. Like, really. Python is a pain in the ass in production environments.

Pytorch has multiple production paths for deployment models without the python interpreter already, like torchscript and libtorch, or more annoying paths like onnx export and onnx runtime. So there is no need to have rust to solve this problem.
Post reply on HN