Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python vs. Rust for Neural Networks

ngoldbaum.github.io

Post reply on HN
Print view View original

21–30 of 149 posts

Re: Python vs. Rust for Neural Networks

#21

Post by atoav »

tanilama wrote: post #12

Nobody writing NN in Python, they are just describing it. For NN or DL in general, the correctness doesn't really lie too much on the code quality level, like ownership Rust people love to talk about. It is more about Numeric stability under/overflow and such. Choice of programming language offers limited help here. I don't think Rust has a killer app for ML/DL community to offer as of now, the focus is vastly differ…

As somebody who programs in both Python and Rust (and likes both languages) I think Rust's place would be parts of the code that have to be fast, and that you want to get right.

Calling Python code from Rust or Rust from Python is totally doable, and there is in my view no reason why you shouldn't use both in the use cases that suit them.

And the speed part is serious. Some guy once asked for the fastest tokenizer in any given language and my naive implementation came second place in his benchmark right after an stripped down and optimized C variant.

So using Rust for speed critical modules and interfacing them from easy to use Python libraries isn't exactly irrational.

Re: Python vs. Rust for Neural Networks

#22

Post by seanmcdirmid »

psv1 wrote: post #10

Neural network libraries (Tensorflow, Pytorch) have a C++ backend and a Python interface. Which is great - you get a performant compiled language as the backend and a flexible user-friendly language as the interface. Rust vs Python is a weird question because in reality no one writes their own neural network with numpy, and no one expects Rust to act like an interpreted language suitable for data science workflows. I…

Or maybe Rust can be compared to Julia if Julia is used for custom machine learning rather than just calling into pre compiled CUDA kernels?

Also, does Rust have a GPU/CUDA backend yet?

Re: Python vs. Rust for Neural Networks

#23

Post by roadbeats »

I’m a newbie in NN topic and feel surprised to hear that noone uses Numpy in actual NN implementations, although it’s written in C++ and highly optimized. Why is that ?

And, how about Gonum (Go equivalent) ?

Finally, I’m currently going through the deeplearning.ai program. I got one week left, and will experiment with building some apps. Which technical stack should I choose ?

Re: Python vs. Rust for Neural Networks

#24

Post by drewm1980 »

tanilama wrote: post #12

Nobody writing NN in Python, they are just describing it. For NN or DL in general, the correctness doesn't really lie too much on the code quality level, like ownership Rust people love to talk about. It is more about Numeric stability under/overflow and such. Choice of programming language offers limited help here. I don't think Rust has a killer app for ML/DL community to offer as of now, the focus is vastly differ…

You're right, and... I'm working professionally using DL for computer vision, for robotics. We spend the majority of our time writing the business logic around the learned parts. To get all of that code bug free and fast is way harder in python that I expect it would be in Rust. Rust's ndarray ecosystem is immature, but Python's static type checking ecosystem is too (no stable mypy stubs for numpy ndarrays), as is Julia's AOT compilation story.

Re: Python vs. Rust for Neural Networks

#25

Post by jimmy_dean »

roadbeats wrote: post #23

I’m a newbie in NN topic and feel surprised to hear that noone uses Numpy in actual NN implementations, although it’s written in C++ and highly optimized. Why is that ? And, how about Gonum (Go equivalent) ? Finally, I’m currently going through the deeplearning.ai program. I got one week left, and will experiment with building some apps. Which technical stack should I choose ?

The main reason numpy isn't used in NN implementations is that it does not, natively speaking, have GPU support. Tensor structures in PyTorch and TensorFlow have the most solid backend support for GPUs (TPUs) and have a good amount of numpy's ndarray capabilities. There is recent work to put numpy on the same footing for deep learning. Check https://github.com/google/jax

Re: Python vs. Rust for Neural Networks

#26

Post by csande17 »

roadbeats wrote: post #23

I’m a newbie in NN topic and feel surprised to hear that noone uses Numpy in actual NN implementations, although it’s written in C++ and highly optimized. Why is that ? And, how about Gonum (Go equivalent) ? Finally, I’m currently going through the deeplearning.ai program. I got one week left, and will experiment with building some apps. Which technical stack should I choose ?

Most software (including Python and Numpy and Go and pretty much every Rust program) runs on your computer's CPU. The CPU is good at running programs with a lot of different instructions and if-statements and loops and stuff.

But for neural networks, people often prefer to use special hardware like graphics cards, since graphics cards are really good at doing relatively simple math on many pieces of data at once. So they create special libraries like TensorFlow that can send commands to the graphics card instead of doing the math on the CPU. (And they don't use Numpy because even though it's highly optimized, it's highly optimized for CPUs, and graphics cards are a lot faster than CPUs at running neural networks.)

Re: Python vs. Rust for Neural Networks

#27

Post by wokwokwok »

kuzehanka wrote: post #20
tanilama wrote: post #12

Nobody writing NN in Python, they are just describing it. For NN or DL in general, the correctness doesn't really lie too much on the code quality level, like ownership Rust people love to talk about. It is more about Numeric stability under/overflow and such. Choice of programming language offers limited help here. I don't think Rust has a killer app for ML/DL community to offer as of now, the focus is vastly differ…

I've had a few Rust lovers come and mention this project to me recently. None of them had any data science or ML experience. None of them knew that Python is just used to define the high level architecture. At the same time, comparatively tedious languages like Rust will never attract data science practitioners. They don't care about the kind of safety it brings, they don't care about improving performance in a compo…

> comparatively tedious languages like Rust will never attract data science practitioners.

Well, fast.ai is using swift now.

... I think it's fair to say 'never say never'.

You're probably right, rust isn't really the sweet spot for this stuff, but its also a case that python has some down sides that are pretty severe, and well acknowledged.

Re: Python vs. Rust for Neural Networks

#28

Post by xxxpupugo »

kuzehanka wrote: post #20
tanilama wrote: post #12

Nobody writing NN in Python, they are just describing it. For NN or DL in general, the correctness doesn't really lie too much on the code quality level, like ownership Rust people love to talk about. It is more about Numeric stability under/overflow and such. Choice of programming language offers limited help here. I don't think Rust has a killer app for ML/DL community to offer as of now, the focus is vastly differ…

I've had a few Rust lovers come and mention this project to me recently. None of them had any data science or ML experience. None of them knew that Python is just used to define the high level architecture. At the same time, comparatively tedious languages like Rust will never attract data science practitioners. They don't care about the kind of safety it brings, they don't care about improving performance in a compo…

CUDA is an important part of the story.

I think the industry is moving to 'MLIR' solution (Yes, there is a Google project called exactly that, but I am referring to the general idea here), where the network is defined and trained in one place, then the weights are exported, delegated to optimized runtime to be executed.

If such trend furthers down, then there will be very little reason to replace Python as the glue layer here. Instead it will become that everything is training in Python -> exported to shared format -> executed in optimized runtime, kind of flow.

Rust's opportunity could be to replace C++ in this case. But do mind that this is also a competitive business, where the computation is pushed further down to the hardware implementations, like TPUv1 and T4 chips etc.

Re: Python vs. Rust for Neural Networks

#29

Post by melling »

wokwokwok wrote: post #27
kuzehanka wrote: post #20

Earlier quoted context omitted.

I've had a few Rust lovers come and mention this project to me recently. None of them had any data science or ML experience. None of them knew that Python is just used to define the high level architecture. At the same time, comparatively tedious languages like Rust will never attract data science practitioners. They don't care about the kind of safety it brings, they don't care about improving performance in a compo…

> comparatively tedious languages like Rust will never attract data science practitioners. Well, fast.ai is using swift now. ... I think it's fair to say 'never say never'. You're probably right, rust isn't really the sweet spot for this stuff, but its also a case that python has some down sides that are pretty severe, and well acknowledged.

I wouldn’t call Swift a tedious language.

With type inference, immutability, etc, Swift is far from tedious:

http://www.h4labs.com/dev/ios/swift_cookbook.html?topic=stri...

http://www.h4labs.com/dev/ios/swift_cookbook.html?topic=dict...

It’s not quite as nice as Python but it’s an enjoyable language.

Re: Python vs. Rust for Neural Networks

#30

Post by PeterisP »

roadbeats wrote: post #23

I’m a newbie in NN topic and feel surprised to hear that noone uses Numpy in actual NN implementations, although it’s written in C++ and highly optimized. Why is that ? And, how about Gonum (Go equivalent) ? Finally, I’m currently going through the deeplearning.ai program. I got one week left, and will experiment with building some apps. Which technical stack should I choose ?

Numpy is at a too low level for applied NN implementations.

If I'm not doing research on new methods but want to build a model for a particular problem using well-known best practices, then all the custom code that my app needs and what I need to write is about the transformation and representation and structure of my particular dataset and task; but things like, for example, optimized backpropagation for a stack of bidirectional LSTM layers are not custom for my app, they're generic - why would I need or want to reimplement them except as a learning exercise?

That'd be like reinventing a bicycle, for generic things like that I'd want to call a library where that code is well-tested and well-optimized (including for GPU usage) by someone else, and that library isn't numpy. Numpy works at the granularity of matrix multiplication ops, but applied ML works at the granularity of whole layers such as self-attention or LSTM or CNN; which perhaps are not that complex conceptually, but do require some attention to implement properly in an optimized way; you can implement them in numpy but you probably shouldn't (unless as a learning exercise).

Post reply on HN