Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python vs. Rust for Neural Networks

ngoldbaum.github.io

Post reply on HN
Print view View original

31–40 of 149 posts

Re: Python vs. Rust for Neural Networks

#31

Post by pjmlp »

kuzehanka wrote: post #20
tanilama wrote: post #12

Nobody writing NN in Python, they are just describing it. For NN or DL in general, the correctness doesn't really lie too much on the code quality level, like ownership Rust people love to talk about. It is more about Numeric stability under/overflow and such. Choice of programming language offers limited help here. I don't think Rust has a killer app for ML/DL community to offer as of now, the focus is vastly differ…

I've had a few Rust lovers come and mention this project to me recently. None of them had any data science or ML experience. None of them knew that Python is just used to define the high level architecture. At the same time, comparatively tedious languages like Rust will never attract data science practitioners. They don't care about the kind of safety it brings, they don't care about improving performance in a compo…

Rust might not be it.

But AOT/JIT compiled languages that can naturally talk to the GPGPU, without 2nd language syndrome, like Julia, Swift, Java and .NET will certainly be more attractive to data science practitioners.

I can already envision those life science guys that migrate to VB.NET when they have outgrown their Excel/VBA code, to start playing with ML.NET.

Re: Python vs. Rust for Neural Networks

#33

Post by frutiger »

melling wrote: post #29
wokwokwok wrote: post #27

Earlier quoted context omitted.

> comparatively tedious languages like Rust will never attract data science practitioners. Well, fast.ai is using swift now. ... I think it's fair to say 'never say never'. You're probably right, rust isn't really the sweet spot for this stuff, but its also a case that python has some down sides that are pretty severe, and well acknowledged.

I wouldn’t call Swift a tedious language. With type inference, immutability, etc, Swift is far from tedious: http://www.h4labs.com/dev/ios/swift_cookbook.html?topic=stri... http://www.h4labs.com/dev/ios/swift_cookbook.html?topic=dict... It’s not quite as nice as Python but it’s an enjoyable language.

Rust has those too, GGGP still considered it tedious.

Re: Python vs. Rust for Neural Networks

#34

Post by kuzehanka »

pjmlp wrote: post #31
kuzehanka wrote: post #20

Earlier quoted context omitted.

I've had a few Rust lovers come and mention this project to me recently. None of them had any data science or ML experience. None of them knew that Python is just used to define the high level architecture. At the same time, comparatively tedious languages like Rust will never attract data science practitioners. They don't care about the kind of safety it brings, they don't care about improving performance in a compo…

Rust might not be it. But AOT/JIT compiled languages that can naturally talk to the GPGPU, without 2nd language syndrome, like Julia, Swift, Java and .NET will certainly be more attractive to data science practitioners. I can already envision those life science guys that migrate to VB.NET when they have outgrown their Excel/VBA code, to start playing with ML.NET.

Python already gets JIT compiled to CUDA[1] and there's an entire funded ecosystem built around python+gpgpu called RAPIDS[2] which is the future of the ML space by most indicators.

I don't see any other language even making a dent in the Python ecosystem without some kind of new killer feature that can't be quickly replicated in Python.

[1] https://numba.pydata.org [2] https://rapids.ai

Re: Python vs. Rust for Neural Networks

#35

Post by antisemiotic »

frutiger wrote: post #33
melling wrote: post #29

Earlier quoted context omitted.

I wouldn’t call Swift a tedious language. With type inference, immutability, etc, Swift is far from tedious: http://www.h4labs.com/dev/ios/swift_cookbook.html?topic=stri... http://www.h4labs.com/dev/ios/swift_cookbook.html?topic=dict... It’s not quite as nice as Python but it’s an enjoyable language.

Rust has those too, GGGP still considered it tedious.

I think the tedious part would be the borrow checker. Also, Rust doesn't have top-level type inference (by design).

Re: Python vs. Rust for Neural Networks

#36

Post by wokwokwok »

melling wrote: post #29
wokwokwok wrote: post #27

Earlier quoted context omitted.

> comparatively tedious languages like Rust will never attract data science practitioners. Well, fast.ai is using swift now. ... I think it's fair to say 'never say never'. You're probably right, rust isn't really the sweet spot for this stuff, but its also a case that python has some down sides that are pretty severe, and well acknowledged.

I wouldn’t call Swift a tedious language. With type inference, immutability, etc, Swift is far from tedious: http://www.h4labs.com/dev/ios/swift_cookbook.html?topic=stri... http://www.h4labs.com/dev/ios/swift_cookbook.html?topic=dict... It’s not quite as nice as Python but it’s an enjoyable language.

> They don't care about the kind of safety it brings, they don't care about improving performance in a component that's idle 99% of the time.

https://www.fast.ai/2019/03/06/fastai-swift/:

> Because Swift for TensorFlow is the first serious effort I’ve seen to incorporate differentiable programming deep in to the heart of a widely used language that is designed from the ground up for performance.

> But Python is not designed to be fast, and it is not designed to be safe. Instead, it is designed to be easy, and flexible. To work around the performance problems of using “pure Python” code, we instead have to use libraries written in other languages (generally C and C++), like numpy, PyTorch, and TensorFlow, which provide Python wrappers. To work around the problem of a lack of type safety, recent versions of Python have added type annotations that optionally allow the programmer to specify the types used in a program. However, Python’s type system is not capable of expressing many types and type relationships, does not do any automated typing, and can not reliably check all types at compile time. Therefore, using types in Python requires a lot of extra code, but falls far short of the level of type safety that other languages can provide.

ie. My point: OP's point is categorically false.

...not that swift is tedious. I <3 swift.

Re: Python vs. Rust for Neural Networks

#37

Post by pilooch »

vs C++-14 ? Indeed most DL is in fact C++. The Pytorch recent C++ API is a must. As professionals in this industry, my colleagues and I have switched to full C++. I'd be interested in advantages of Rust vs C++ instead of Python (which truely in terms of performances is C in the background).

Re: Python vs. Rust for Neural Networks

#38

Post by kuzehanka »

wokwokwok wrote: post #36
melling wrote: post #29

Earlier quoted context omitted.

I wouldn’t call Swift a tedious language. With type inference, immutability, etc, Swift is far from tedious: http://www.h4labs.com/dev/ios/swift_cookbook.html?topic=stri... http://www.h4labs.com/dev/ios/swift_cookbook.html?topic=dict... It’s not quite as nice as Python but it’s an enjoyable language.

> They don't care about the kind of safety it brings, they don't care about improving performance in a component that's idle 99% of the time. https://www.fast.ai/2019/03/06/fastai-swift/ : > Because Swift for TensorFlow is the first serious effort I’ve seen to incorporate differentiable programming deep in to the heart of a widely used language that is designed from the ground up for performance. > But Python is not…

...But fast.ai is a Python library. Partially written in swift.

Validating the original point that nothing will replace python for DL applications any time soon but middleware will continue to be implemented in c++/rust/swift/whatever you fancy.

S4TF isn't the first and certainly not the last end to end non-python DL stack. It might be worth highlighting as an example if it ever reaches mindshare above the noise floor amongst those stacks.

Re: Python vs. Rust for Neural Networks

#39

Post by chillee »

roadbeats wrote: post #23

I’m a newbie in NN topic and feel surprised to hear that noone uses Numpy in actual NN implementations, although it’s written in C++ and highly optimized. Why is that ? And, how about Gonum (Go equivalent) ? Finally, I’m currently going through the deeplearning.ai program. I got one week left, and will experiment with building some apps. Which technical stack should I choose ?

There's 3 fundamental gaps between Numpy and NN libraries (other commenters have jointly mentioned 2/3)

1. Numpy doesn't run on GPU.

2. Numpy isn't high level enough for NN building.

3. Numpy doesn't have auto differentiation.

Other options solve some of these - Autograd solves 3, Jax solves 1 and 3, etc.

But if you want all 3 then you want to use Pytorch.

Re: Python vs. Rust for Neural Networks

#40

Post by chillee »

wokwokwok wrote: post #27
kuzehanka wrote: post #20

Earlier quoted context omitted.

I've had a few Rust lovers come and mention this project to me recently. None of them had any data science or ML experience. None of them knew that Python is just used to define the high level architecture. At the same time, comparatively tedious languages like Rust will never attract data science practitioners. They don't care about the kind of safety it brings, they don't care about improving performance in a compo…

> comparatively tedious languages like Rust will never attract data science practitioners. Well, fast.ai is using swift now. ... I think it's fair to say 'never say never'. You're probably right, rust isn't really the sweet spot for this stuff, but its also a case that python has some down sides that are pretty severe, and well acknowledged.

Just because fast.ai has some investment in Swift does not mean that S4TF has attracted mind share. The bulk of fast.ai is still taught on Pytorch.
Post reply on HN