Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Flashlight: Fast and flexible machine learning in C++

ai.facebook.com

Post reply on HN
Print view View original

51–60 of 96 posts

Re: Flashlight: Fast and flexible machine learning in C++

#51

Post by davnicwil »

dwrodri wrote: post #21

This seems really cool, but I don't get why they would pour work into this while simultaneously building a C++ front-end for PyTorch[1]. Per the blog post, both frameworks have the goal of empowering ML researchers to iterate on ML models in such a way that it becomes easier to reason about performance than it would be talking to a bunch of dynamically linked object files behind an interpreter. Facebook is a huge com…

> huge company with lots of money... [yet] management that thinks two different C++ ML frameworks with the same goals is a good idea

I don't see a contradiction here, in fact it makes sense.

You have some goals, and a couple of promising approaches. It's hard to say which will work better, but there's enough budget and people to work on them to just try them both and see.

I've heard anecdotes of similar strategies at banks, who have sufficient budget to hire two parallel teams to build literally the same product, sometimes without even knowing about each other. At the end, the one that ends up being faster/better gets used.

I guess it's like a microcosm of free market competition within an org, as opposed to top down planning.

Re: Flashlight: Fast and flexible machine learning in C++

#52

Post by antpls »

dynamite-ready wrote: post #32

Does anyone else think that C++ makes more sense for ML work than Python? I'd been thinking so for years. Both for deployment/performance and data wrangling purposes.

In my opinion, no, it makes no sense to write ML in C++ :

- Python allows for higher level description of algorithms, which means researchers can focus more on the ML stuff and less on low level details.

- There is no performance gain in going from Python to C++, because in both cases the models are compiled to specific binary formats to be executed on dedicated hardwares. TensorFlow enables accelerators not only for training, but also for data transformations and preprocessing.

Re: Flashlight: Fast and flexible machine learning in C++

#53

Post by stephc_int13 »

Given the very high computing requirements of Machine Learning, I've always be perplexed by the seemingly widespread and unquestionable preference of Python over native code (typically C/C++).

I guess performance was considered less critical than clarity/flexibility. But it seems that people are discovering that complex code tend to be hard to read/modify no matter the language...

Re: Flashlight: Fast and flexible machine learning in C++

#54

Post by andrew_v4 »

stephc_int13 wrote: post #53

Given the very high computing requirements of Machine Learning, I've always be perplexed by the seemingly widespread and unquestionable preference of Python over native code (typically C/C++). I guess performance was considered less critical than clarity/flexibility. But it seems that people are discovering that complex code tend to be hard to read/modify no matter the language...

Python (in common ML frameworks) is really just wrapping well optimized native code. The overhead is very minimal. The advantage here is about easy access to the internals. I don't know that any material speedup is expected just because its written in C++

Re: Flashlight: Fast and flexible machine learning in C++

#55

Post by Joky »

stephc_int13 wrote: post #53

Given the very high computing requirements of Machine Learning, I've always be perplexed by the seemingly widespread and unquestionable preference of Python over native code (typically C/C++). I guess performance was considered less critical than clarity/flexibility. But it seems that people are discovering that complex code tend to be hard to read/modify no matter the language...

Python isn't really driving the compute intensive part of ML actually, whether it's JAX, PyTorch, or TensorFlow the code is really mostly native. Convolution are implemented by hand in highly optimized libraries (Intel MKL-DNN, Nvidia cuDNN) and the Python glue is really just a light "dispatcher".

A lot of it is also asynchronous for performance: the Python code just enqueues more work to a queue which some native C++ code processes. For TensorFlow the Python code traces an entire computation graph that is stored a protobuf and then executed by a C++ native stack, potentially remotely/distributed. Serving ML with TensorFlow does not involve any Python code in many scenarios.

Python is still quite useful for scientist to quickly glue everything together, and to describe their dataset, or when they collect result and need to produce graphs or other data analyses.

Re: Flashlight: Fast and flexible machine learning in C++

#56

Post by stephc_int13 »

andrew_v4 wrote: post #54
stephc_int13 wrote: post #53

Given the very high computing requirements of Machine Learning, I've always be perplexed by the seemingly widespread and unquestionable preference of Python over native code (typically C/C++). I guess performance was considered less critical than clarity/flexibility. But it seems that people are discovering that complex code tend to be hard to read/modify no matter the language...

Python (in common ML frameworks) is really just wrapping well optimized native code. The overhead is very minimal. The advantage here is about easy access to the internals. I don't know that any material speedup is expected just because its written in C++

I've worked at some point on a commercial game engine written in C++ at the core but with many Lua components and API, for convenience.

The reasoning wat the same, all the heavy lifting done with fast native code, and everything written in Lua was mostly glue code without real performance impact.

Turns out the engine was slow and difficult to maintain because of the many interfaces. They ditched it a few years later...

I think you could say the same thing about javascript on the web, this is mostly glue code, all the heavy lifting is done by optimized C++ or Rust code in the browser. And yet it is difficult to run a web mail client if your computer is a bit old...

This is probably related in some ways to Amdahl's law https://en.wikipedia.org/wiki/Amdahl%27s_law

Re: Flashlight: Fast and flexible machine learning in C++

#57

Post by throwaway5849 »

whimsicalism wrote: post #13
angry_octet wrote: post #9

Earlier quoted context omitted.

Yes, I could see the construction, I just can't think why they wouldn't anticipate the problems it could cause down the track.

What? Nobody in their right mind thinks "fleshlight" immediately upon hearing the word "flashlight" Is this some sort of performance art comment?

Reviving my old throwaway account to confirm that I indeed thought of "fleshlight" immediately after reading the announcement. Keep in mind that the average coder is far more likely to read Reddit daily (where this reference comes up surprisingly often) than to do things that require a flashlight (own a house somewhere remote, etc)

Re: Flashlight: Fast and flexible machine learning in C++

#58

Post by throwaway5849 »

snovv_crash wrote: post #31
angry_octet wrote: post #9

Earlier quoted context omitted.

Yes, I could see the construction, I just can't think why they wouldn't anticipate the problems it could cause down the track.

You spend too much time on the internet.

In fairness, so does the average programmer.

Re: Flashlight: Fast and flexible machine learning in C++

#59

Post by kwertyoowiyop »

dwrodri wrote: post #21

This seems really cool, but I don't get why they would pour work into this while simultaneously building a C++ front-end for PyTorch[1]. Per the blog post, both frameworks have the goal of empowering ML researchers to iterate on ML models in such a way that it becomes easier to reason about performance than it would be talking to a bunch of dynamically linked object files behind an interpreter. Facebook is a huge com…

Oh sweet summer child, it’s so refreshing that you think big famous companies always act logically!

Re: Flashlight: Fast and flexible machine learning in C++

#60

Post by person_of_color »

codebolt wrote: post #43
dynamite-ready wrote: post #32

Does anyone else think that C++ makes more sense for ML work than Python? I'd been thinking so for years. Both for deployment/performance and data wrangling purposes.

Absolutely. In 2010 I used FANN for my Msc thesis research, and found it pretty easy to make my own little training sim for stock price data on top of it. Haven't done any ML work since, but I always scratched my head over how Python became the most popular language for this domain.

stock price ml? why aren’t you rich?
Post reply on HN