Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python vs. Rust for Neural Networks

ngoldbaum.github.io

Post reply on HN
Print view View original

121–130 of 149 posts

Re: Python vs. Rust for Neural Networks

#121

Post by likeabbas »

kuzehanka wrote: post #65
danielscrubs wrote: post #51

Earlier quoted context omitted.

I am a data scientist and I care. The time when you could just do proof of concepts or a PowerPoint presentation is long behind us. So now we have to start to take it into production, which means we get the exact same problems as SE has always had. Iff Rust helps us take it into production we will use it. But it’s a lot of land to cover to reach Pythons libraries so I’m not holding my breath. That said, Pythons perfo…

I must be missing something. Modern data science workloads involve fanning out data and code across dozens to hundreds of nodes. The bottlenecks, in order, are: inter-node comms, gpu/compute, on-disk shuffling, serialisation, pipeline starvation, and finally the runtime. Why worry about optimising the very top of the perf pyramid which will make the least difference? Why worry if you spent 1ms pushing data to numpy w…

A big push for NN is to get them running in real time on local GPUs so we can make AI cars and other AI tech a reality. 2500ms could be life and death in many scenarios

Re: Python vs. Rust for Neural Networks

#122

Post by ddragon »

MiroF wrote: post #116
ddragon wrote: post #111

Earlier quoted context omitted.

Do you have an example of a tensor library that keep track of shapes and detect mismatches at compile time? I had the impression that even in static languages having tensors with the exact shape as a parameter would stress the compiler, forcing it to compile many versions of every function for every possible size combination, and the output of a function could very well have a non deterministic or multiple possible s…

If you explain a little more of what you mean, I might be able to respond more effectively. > I had the impression that even in static languages having tensors with the exact shape as a parameter would stress the compiler, forcing it to compile many versions of every function for every possible size combination, and the output of a function could very well have a non deterministic or multiple possible shapes (for exa…

You understood it correctly. Named dimensions is certainly a good idea even in dynamic languages as a way of documentation and runtime errors that actually make sense (instead of stuff like "expected shape (24, 128, 32) but got (32, 128, 24)"). I hope it catches on.

Re: Python vs. Rust for Neural Networks

#123

Post by MiroF »

ddragon wrote: post #122
MiroF wrote: post #116

Earlier quoted context omitted.

If you explain a little more of what you mean, I might be able to respond more effectively. > I had the impression that even in static languages having tensors with the exact shape as a parameter would stress the compiler, forcing it to compile many versions of every function for every possible size combination, and the output of a function could very well have a non deterministic or multiple possible shapes (for exa…

You understood it correctly. Named dimensions is certainly a good idea even in dynamic languages as a way of documentation and runtime errors that actually make sense (instead of stuff like "expected shape (24, 128, 32) but got (32, 128, 24)"). I hope it catches on.

But combined with static checking, it could be very very powerful. Definitely agree re: the power even in dynamic languages, I use namedtensor for almost all my personal development now (less so in work because of interop issues)

Re: Python vs. Rust for Neural Networks

#124

Post by kognate »

MiroF wrote: post #101
skohan wrote: post #86

Earlier quoted context omitted.

It's a fair criticism of Swift that it is not well supported across platforms. The story is getting better, but it is still miles behind Rust in this regard. I am a huge fan of Swift as a language, but I am very critical of the tooling. > Modern Rust isn't difficult to use. This is becoming a really tired meme from detractors. I beg to differ. I've been programming professionally for over a decade, and I have shipped…

The tooling for Swift 4 TF is not anywhere near satisfactory. I can't even seem to get it on my computer without installing XCode (and that's on an apple computer) I should be able to pull a docker image and have s4tf immediately at my fingertips

Seems the Fast.ai forums have got you covered here. https://forums.fast.ai/t/swift-for-tensorflow-using-gpu-with... describes how to do it, while not completely 'at my fingertips' it's pretty close. and Googles repo at https://github.com/google/swift-jupyter has a clean setup, too.

Re: Python vs. Rust for Neural Networks

#125

Post by MiroF »

kognate wrote: post #124
MiroF wrote: post #101

Earlier quoted context omitted.

The tooling for Swift 4 TF is not anywhere near satisfactory. I can't even seem to get it on my computer without installing XCode (and that's on an apple computer) I should be able to pull a docker image and have s4tf immediately at my fingertips

Seems the Fast.ai forums have got you covered here. https://forums.fast.ai/t/swift-for-tensorflow-using-gpu-with... describes how to do it, while not completely 'at my fingertips' it's pretty close. and Googles repo at https://github.com/google/swift-jupyter has a clean setup, too.

I've come across that first link before, but remember I had some issues.

The two problems are a. I don't have a GPU locally b. I would rather have a compiler than just a REPL/jupyter

Re: Python vs. Rust for Neural Networks

#127

Post by lreichold »

MiroF wrote: post #101
skohan wrote: post #86

Earlier quoted context omitted.

It's a fair criticism of Swift that it is not well supported across platforms. The story is getting better, but it is still miles behind Rust in this regard. I am a huge fan of Swift as a language, but I am very critical of the tooling. > Modern Rust isn't difficult to use. This is becoming a really tired meme from detractors. I beg to differ. I've been programming professionally for over a decade, and I have shipped…

The tooling for Swift 4 TF is not anywhere near satisfactory. I can't even seem to get it on my computer without installing XCode (and that's on an apple computer) I should be able to pull a docker image and have s4tf immediately at my fingertips

Not local, but you can start using S4TF immediately via Google Colab.

https://colab.research.google.com/github/tensorflow/swift/bl...

Re: Python vs. Rust for Neural Networks

#128

Post by hsaliak »

kuzehanka wrote: post #34
pjmlp wrote: post #31

Earlier quoted context omitted.

Rust might not be it. But AOT/JIT compiled languages that can naturally talk to the GPGPU, without 2nd language syndrome, like Julia, Swift, Java and .NET will certainly be more attractive to data science practitioners. I can already envision those life science guys that migrate to VB.NET when they have outgrown their Excel/VBA code, to start playing with ML.NET.

Python already gets JIT compiled to CUDA[1] and there's an entire funded ecosystem built around python+gpgpu called RAPIDS[2] which is the future of the ML space by most indicators. I don't see any other language even making a dent in the Python ecosystem without some kind of new killer feature that can't be quickly replicated in Python. [1] https://numba.pydata.org [2] https://rapids.ai

You beat me to making this point.

I don't see anything in the article's python code that the numba's jit decorator cannot handle. When numba works (it's rapidly improving), it's seriously impressive.

For this particular case, you should be able to get really good performance without sacrificing readability.

Also Jax - https://github.com/google/jax

Re: Python vs. Rust for Neural Networks

#129

Post by kahnjw »

danielscrubs wrote: post #120
kahnjw wrote: post #108

Earlier quoted context omitted.

This is just not true. The python runtime is not the bottleneck. DL frameworks are DSLs written on top of piles of highly optimized C++ code that is executed as independently from the python runtime as possible. Optimizing the python or swapping it out for some other language is not going to buy you anything except a ton of work. We can argue about using rust to implement the lower level ops instead of c++. That migh…

Yes. Let’s say you want certain features of a voice sample. You need to do that feature engineering every time before you send it to the model. Doesn’t it make sense to do it in C++ or Rust? This is currently already done. So if you already are starting to do parts of the feature engineering in Rust why not continue? Yeah it’s not reasonable right now because Python has the best ecosystem. But that will not always be…

I can’t exactly tell what you mean but I think you’re confusing two levels of abstraction here. C++ (or rust) and python already work in harmony to make training efficient.

1. In tensorflow and similar frameworks the Python runtime is used to compose highly optimized operations to create a trainable graph. 2. C++ is used to implement those highly optimized ops. If you have some novel feature engineering and you need better throughput performance than a pure python op can give you, you’d implement the most general viable c++ (or rust) op and then use that wrapped op in python.

This is how large companies scale machine learning in general, though this applies to all ops not just feature engineering specific ones.

There is no way that Instagram is using a pure python image processing lib to prep images for their porn detection models. That would cost too much money and take way too much time. Instead they almost certainly wrap some c++ in some python and move on to more important things.

Re: Python vs. Rust for Neural Networks

#130

Post by cwyers »

skohan wrote: post #60
xxxpupugo wrote: post #28

Earlier quoted context omitted.

CUDA is an important part of the story. I think the industry is moving to 'MLIR' solution (Yes, there is a Google project called exactly that, but I am referring to the general idea here), where the network is defined and trained in one place, then the weights are exported, delegated to optimized runtime to be executed. If such trend furthers down, then there will be very little reason to replace Python as the glue l…

Personally I hope for alternatives because I don't find Python that nice to work with compared to other languages. It's not awful, but I really miss having type and nullability errors detected by the compiler and reported to me with meaningful error messages. Also there are a few weird things with the Python workflow, like dealing with Python2 vs Python3, and generally having to run everything in the context of a vir…

There are alternatives -- Julia and R both have bindings for Keras, which gets you Tensorflow, CNTK and Theano. (I think there are bindings for Tensorflow directly from R and Julia as well.) Once you have a trained model, it doesn't really matter from the production standpoint how you trained it.
Post reply on HN