Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python vs. Rust for Neural Networks

ngoldbaum.github.io

Post reply on HN
Print view View original

101–110 of 149 posts

Re: Python vs. Rust for Neural Networks

#101

Post by MiroF »

skohan wrote: post #86
echelon wrote: post #83

Earlier quoted context omitted.

Swift is not fun on non-Mac platforms. It feels a lot like Google's Dart in terms of how it was positioned and advocated. Modern Rust isn't difficult to use. This is becoming a really tired meme from detractors. The compiler is incredibly helpful, non-lexical lifetimes are a thing, and unless you're doing a lot of sharing and parallelism, you can avoid many borrow checker problems until you learn RAII.

It's a fair criticism of Swift that it is not well supported across platforms. The story is getting better, but it is still miles behind Rust in this regard. I am a huge fan of Swift as a language, but I am very critical of the tooling. > Modern Rust isn't difficult to use. This is becoming a really tired meme from detractors. I beg to differ. I've been programming professionally for over a decade, and I have shipped…

The tooling for Swift 4 TF is not anywhere near satisfactory. I can't even seem to get it on my computer without installing XCode (and that's on an apple computer)

I should be able to pull a docker image and have s4tf immediately at my fingertips

Re: Python vs. Rust for Neural Networks

#103

Post by danielscrubs »

MiroF wrote: post #91
danielscrubs wrote: post #51

Earlier quoted context omitted.

I am a data scientist and I care. The time when you could just do proof of concepts or a PowerPoint presentation is long behind us. So now we have to start to take it into production, which means we get the exact same problems as SE has always had. Iff Rust helps us take it into production we will use it. But it’s a lot of land to cover to reach Pythons libraries so I’m not holding my breath. That said, Pythons perfo…

You are a data scientist who seems to lack an understanding of how deep models are productionized.. I do so not unfrequently and I don't see how rust bindings would help me at all

or maybe, just maybe, you know nothing of optimizations, so you just go: impossible!

Re: Python vs. Rust for Neural Networks

#104

Post by MiroF »

asdjlkadsjklads wrote: post #64
marmaduke wrote: post #16

Rust seems more suitable for implementing the next OpenBLAS. While Julia's single language mantra is great, as long as things like Python exist, there will be a need for C/C++/Rust.

Yea, this whole discussion feels weird to me. Different use cases. I love Rust (and dislike Py lol), but from everything i hear a highly dynamic frontend (like Py) has little downsides to authors of ML/etc. All of the hotpaths are in other already because Python is so slow. The only downside i've seen is sometimes the programmer will want more safety. In such a scenario Rust for the "frontend" would be very useful. S…

A dynamic language can be super frustrating to develop with because you have to keep tensor dimensions memorized in your head or in comments

Re: Python vs. Rust for Neural Networks

#105

Post by kuzehanka »

danielscrubs wrote: post #88
kuzehanka wrote: post #65

Earlier quoted context omitted.

I must be missing something. Modern data science workloads involve fanning out data and code across dozens to hundreds of nodes. The bottlenecks, in order, are: inter-node comms, gpu/compute, on-disk shuffling, serialisation, pipeline starvation, and finally the runtime. Why worry about optimising the very top of the perf pyramid which will make the least difference? Why worry if you spent 1ms pushing data to numpy w…

Every small thing counts when you have big data which is exactly why you need performance everywhere, if Rust can help with that I don’t mind switching my team to that. The problem are usually when you do novel feature engineering not the actual model training. But I was a C++ dev before checking the assembly for performance optimization so I guess I have more wiggle room to see when things are not up to snuff. If I…

Novel feature engineering? Like this? https://towardsdatascience.com/python-performance-and-gpus-1...

Re: Python vs. Rust for Neural Networks

#106

Post by MiroF »

timClicks wrote: post #43

I think it's quite impressive actually that someone can pick up Rust and manage to out-perform Numpy in their first project. BLAS implementations are decades-long exercises in optimization. In my own experience, Rust has been excellent for the more boring side of data science - churning through TBs of input data.

This is a common phenomenon (as anyone who has tried to rewrite the standard library buffer cache can tell you) - the reason is that these algorithms are oftentimes optimized to perform decently on the very very worst cases which means that they'll be slightly slower overall

Re: Python vs. Rust for Neural Networks

#107

Post by MiroF »

danielscrubs wrote: post #103
MiroF wrote: post #91

Earlier quoted context omitted.

You are a data scientist who seems to lack an understanding of how deep models are productionized.. I do so not unfrequently and I don't see how rust bindings would help me at all

or maybe, just maybe, you know nothing of optimizations, so you just go: impossible!

the optimization you are describing is premature - you don't need rust to productionize your models and in most cases you don't even need to be coding in a low-level language at all.

Re: Python vs. Rust for Neural Networks

#108

Post by kahnjw »

danielscrubs wrote: post #88
kuzehanka wrote: post #65

Earlier quoted context omitted.

I must be missing something. Modern data science workloads involve fanning out data and code across dozens to hundreds of nodes. The bottlenecks, in order, are: inter-node comms, gpu/compute, on-disk shuffling, serialisation, pipeline starvation, and finally the runtime. Why worry about optimising the very top of the perf pyramid which will make the least difference? Why worry if you spent 1ms pushing data to numpy w…

Every small thing counts when you have big data which is exactly why you need performance everywhere, if Rust can help with that I don’t mind switching my team to that. The problem are usually when you do novel feature engineering not the actual model training. But I was a C++ dev before checking the assembly for performance optimization so I guess I have more wiggle room to see when things are not up to snuff. If I…

This is just not true. The python runtime is not the bottleneck. DL frameworks are DSLs written on top of piles of highly optimized C++ code that is executed as independently from the python runtime as possible. Optimizing the python or swapping it out for some other language is not going to buy you anything except a ton of work. We can argue about using rust to implement the lower level ops instead of c++. That might be sensible though not from a perspective of performance.

In a "serving environment" where latency actually matters there are already a plethora of solutions for running models directly from a C++ binary, no python needed.

This is a solved problem and people trying to re-invent the wheel with "optimized" implementations are going to be disappointed when they realize their solution doesn't improve anything.

Re: Python vs. Rust for Neural Networks

#109

Post by echelon »

MiroF wrote: post #94
echelon wrote: post #82

Earlier quoted context omitted.

I think this project has immediate merits when it comes to productionizing small, sparse networks for running on the CPU. I do research and prototyping in Python, but I have to deploy on mobile devices. I was going to roll my own implementation, but now that this exists, it's something I'm going to look into.

Load the model from torch script? In C++? Don't see why Rust made this possible.

Ideally we'd have native Rust loading of tensorflow, pytorch, etc. models. I've hand-coded a few, but it was pretty monotonous.

Re: Python vs. Rust for Neural Networks

#110

Post by kahnjw »

corndoge wrote: post #72
kuzehanka wrote: post #65

Earlier quoted context omitted.

I must be missing something. Modern data science workloads involve fanning out data and code across dozens to hundreds of nodes. The bottlenecks, in order, are: inter-node comms, gpu/compute, on-disk shuffling, serialisation, pipeline starvation, and finally the runtime. Why worry about optimising the very top of the perf pyramid which will make the least difference? Why worry if you spent 1ms pushing data to numpy w…

Good lord, hopefully latency isn't 2.5 seconds!

Latency in training literally does not matter. You care about throughput. In serving, where latency matters, most DL frameworks allow you to serve the model from a highly optimized C++ binary, no python needed.

The poster you are replying to is 100% correct.

Post reply on HN