Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python vs. Rust for Neural Networks

ngoldbaum.github.io

Post reply on HN
Print view View original

91–100 of 149 posts

Re: Python vs. Rust for Neural Networks

#91

Post by MiroF »

danielscrubs wrote: post #51
kuzehanka wrote: post #20

Earlier quoted context omitted.

I've had a few Rust lovers come and mention this project to me recently. None of them had any data science or ML experience. None of them knew that Python is just used to define the high level architecture. At the same time, comparatively tedious languages like Rust will never attract data science practitioners. They don't care about the kind of safety it brings, they don't care about improving performance in a compo…

I am a data scientist and I care. The time when you could just do proof of concepts or a PowerPoint presentation is long behind us. So now we have to start to take it into production, which means we get the exact same problems as SE has always had. Iff Rust helps us take it into production we will use it. But it’s a lot of land to cover to reach Pythons libraries so I’m not holding my breath. That said, Pythons perfo…

You are a data scientist who seems to lack an understanding of how deep models are productionized..

I do so not unfrequently and I don't see how rust bindings would help me at all

Re: Python vs. Rust for Neural Networks

#92

Post by danielscrubs »

asdjlkadsjklads wrote: post #64
marmaduke wrote: post #16

Rust seems more suitable for implementing the next OpenBLAS. While Julia's single language mantra is great, as long as things like Python exist, there will be a need for C/C++/Rust.

Yea, this whole discussion feels weird to me. Different use cases. I love Rust (and dislike Py lol), but from everything i hear a highly dynamic frontend (like Py) has little downsides to authors of ML/etc. All of the hotpaths are in other already because Python is so slow. The only downside i've seen is sometimes the programmer will want more safety. In such a scenario Rust for the "frontend" would be very useful. S…

Novel feature engineering is becoming a must. Do you reach for Python or C++? Do you want multiple languages if one was enough?

Re: Python vs. Rust for Neural Networks

#93

Post by MiroF »

gbrown wrote: post #77
kuzehanka wrote: post #65

Earlier quoted context omitted.

I must be missing something. Modern data science workloads involve fanning out data and code across dozens to hundreds of nodes. The bottlenecks, in order, are: inter-node comms, gpu/compute, on-disk shuffling, serialisation, pipeline starvation, and finally the runtime. Why worry about optimising the very top of the perf pyramid which will make the least difference? Why worry if you spent 1ms pushing data to numpy w…

Not everyone operates at that scale, and not every data science workload is DNN based I agree with your general point, however, but the role I'd hope for with Rust is not optimizing the top level, but replacing the mountains of C++ with something safer and equally performant.

But the title of this post is Python vs Rust, not C++ vs Rust. Maybe BLAS could be made safer but i don't think that's what's happening here

Re: Python vs. Rust for Neural Networks

#94

Post by MiroF »

echelon wrote: post #82
kuzehanka wrote: post #20

Earlier quoted context omitted.

I've had a few Rust lovers come and mention this project to me recently. None of them had any data science or ML experience. None of them knew that Python is just used to define the high level architecture. At the same time, comparatively tedious languages like Rust will never attract data science practitioners. They don't care about the kind of safety it brings, they don't care about improving performance in a compo…

I think this project has immediate merits when it comes to productionizing small, sparse networks for running on the CPU. I do research and prototyping in Python, but I have to deploy on mobile devices. I was going to roll my own implementation, but now that this exists, it's something I'm going to look into.

Load the model from torch script? In C++?

Don't see why Rust made this possible.

Re: Python vs. Rust for Neural Networks

#95

Post by MiroF »

xxxpupugo wrote: post #28
kuzehanka wrote: post #20

Earlier quoted context omitted.

I've had a few Rust lovers come and mention this project to me recently. None of them had any data science or ML experience. None of them knew that Python is just used to define the high level architecture. At the same time, comparatively tedious languages like Rust will never attract data science practitioners. They don't care about the kind of safety it brings, they don't care about improving performance in a compo…

CUDA is an important part of the story. I think the industry is moving to 'MLIR' solution (Yes, there is a Google project called exactly that, but I am referring to the general idea here), where the network is defined and trained in one place, then the weights are exported, delegated to optimized runtime to be executed. If such trend furthers down, then there will be very little reason to replace Python as the glue l…

Python should and will be replaced, but not at all for any of the reasons mentioned in this thread.

A good ML language is going to need smart and static typing. I am so tired of having to run a whole network just to figure out that there's a dimension mismatch because I forgot to take a transpose somewhere - there is essentially no reason that tensor shapes can't just be inferred and these errors caught pre-runtime.

Re: Python vs. Rust for Neural Networks

#96

Post by MiroF »

pjmlp wrote: post #31
kuzehanka wrote: post #20

Earlier quoted context omitted.

I've had a few Rust lovers come and mention this project to me recently. None of them had any data science or ML experience. None of them knew that Python is just used to define the high level architecture. At the same time, comparatively tedious languages like Rust will never attract data science practitioners. They don't care about the kind of safety it brings, they don't care about improving performance in a compo…

Rust might not be it. But AOT/JIT compiled languages that can naturally talk to the GPGPU, without 2nd language syndrome, like Julia, Swift, Java and .NET will certainly be more attractive to data science practitioners. I can already envision those life science guys that migrate to VB.NET when they have outgrown their Excel/VBA code, to start playing with ML.NET.

Your last paragraph is my nightmare

Re: Python vs. Rust for Neural Networks

#97

Post by MiroF »

kuzehanka wrote: post #34
pjmlp wrote: post #31

Earlier quoted context omitted.

Rust might not be it. But AOT/JIT compiled languages that can naturally talk to the GPGPU, without 2nd language syndrome, like Julia, Swift, Java and .NET will certainly be more attractive to data science practitioners. I can already envision those life science guys that migrate to VB.NET when they have outgrown their Excel/VBA code, to start playing with ML.NET.

Python already gets JIT compiled to CUDA[1] and there's an entire funded ecosystem built around python+gpgpu called RAPIDS[2] which is the future of the ML space by most indicators. I don't see any other language even making a dent in the Python ecosystem without some kind of new killer feature that can't be quickly replicated in Python. [1] https://numba.pydata.org [2] https://rapids.ai

Swift and static type checking and compiler analysis might be that language and feature combo.

Re: Python vs. Rust for Neural Networks

#99

Post by maayank »

atoav wrote: post #21
tanilama wrote: post #12

Nobody writing NN in Python, they are just describing it. For NN or DL in general, the correctness doesn't really lie too much on the code quality level, like ownership Rust people love to talk about. It is more about Numeric stability under/overflow and such. Choice of programming language offers limited help here. I don't think Rust has a killer app for ML/DL community to offer as of now, the focus is vastly differ…

As somebody who programs in both Python and Rust (and likes both languages) I think Rust's place would be parts of the code that have to be fast, and that you want to get right. Calling Python code from Rust or Rust from Python is totally doable, and there is in my view no reason why you shouldn't use both in the use cases that suit them. And the speed part is serious. Some guy once asked for the fastest tokenizer in…

> Calling Python code from Rust or Rust from Python is totally doable, and there is in my view no reason why you shouldn't use both in the use cases that suit them.

What would be the best route to do so in your experience?

Re: Python vs. Rust for Neural Networks

#100

Post by MiroF »

chillee wrote: post #89
wokwokwok wrote: post #47

Earlier quoted context omitted.

The parent comment literally said data science practitioners don't care about speed or safety because the GPU is where all the real work happens; that's false, I've provided an example of it being false from a respected party. What do you want me to say? eh, I give up. Believe whatever you want to believe.

I'm not saying that no data science practitioners care about speed and safety, and its easy for me to come up with cases where they should/do care. My point is that even fast.ai still views S4TF as somewhat niche, and that data science practitioners as a whole still don't care.

I am a practitioner, I care, but everyone on this thread seems to be distracted by the red herring of speed and efficiency.

None of these languages are going to be speedier in the GPU. But python has serious design problems that manifest itself when developing a large project.

You can only go so far with dynamic typing

Post reply on HN