Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Flashlight: Fast and flexible machine learning in C++

ai.facebook.com

Post reply on HN
Print view View original

81–90 of 96 posts

Re: Flashlight: Fast and flexible machine learning in C++

#81

Post by Narew »

dwrodri wrote: post #21

This seems really cool, but I don't get why they would pour work into this while simultaneously building a C++ front-end for PyTorch[1]. Per the blog post, both frameworks have the goal of empowering ML researchers to iterate on ML models in such a way that it becomes easier to reason about performance than it would be talking to a bunch of dynamically linked object files behind an interpreter. Facebook is a huge com…

btw, swift for tensorflow have been discontinued https://github.com/tensorflow/swift

Re: Flashlight: Fast and flexible machine learning in C++

#82

Post by pilooch »

suyash wrote: post #35

Wonder why they didn't leverage TorchScript ? However this is a welcome news in ML Community. Would love to see this library break away from NVIDIA CUDA dependency and optimize for Apple M1 chips

Torchscript is very complicated. Its internal API was not truely documented for external use last time I checked it. You can't even export to torchscript from libtorch C++, it only works from Python AFAIK.

Re: Flashlight: Fast and flexible machine learning in C++

#83

Post by stephc_int13 »

pjmlp wrote: post #77
stephc_int13 wrote: post #56

Earlier quoted context omitted.

I've worked at some point on a commercial game engine written in C++ at the core but with many Lua components and API, for convenience. The reasoning wat the same, all the heavy lifting done with fast native code, and everything written in Lua was mostly glue code without real performance impact. Turns out the engine was slow and difficult to maintain because of the many interfaces. They ditched it a few years later.…

That was the same reasoning why Unreal dropped UnrealScript. https://www.gamasutra.com/view/news/213647/Epics_Tim_Sweeney... However a couple of years later and Epic is bringing scripting back with Verse https://twitter.com/saji8k/status/1339709691564179464?s=20 Because the problem is not the scripting, but how it is done. I imagine that the engine you mention did not use any kind of compiler for Lua, e.g. LuaJIT, no…

The engine used LuaJIT, but this is not available on iOS.

The code was a mess, absolutely no batching, quite the contrary.

Re: Flashlight: Fast and flexible machine learning in C++

#84

Post by cma »

stephc_int13 wrote: post #53

Given the very high computing requirements of Machine Learning, I've always be perplexed by the seemingly widespread and unquestionable preference of Python over native code (typically C/C++). I guess performance was considered less critical than clarity/flexibility. But it seems that people are discovering that complex code tend to be hard to read/modify no matter the language...

Notebooks may also be a big player, being able to visualize experimental changes rapidly.

Re: Flashlight: Fast and flexible machine learning in C++

#85

Post by whimsicalism »

sp-matcha wrote: post #69
whimsicalism wrote: post #23

Earlier quoted context omitted.

Flashlight is much lower level and gives more fine-grained performance control. For instance, I don't think there is really any way to do real-time speech recognition that is fast with PyTorch because of how it is architected. From my understanding, Tensor Comprehensions and Halide are both very tentative research projects. > not having it's direction beholden to the whims of Alphabet (see: Swift for Tensorflow). I d…

Adding to the above - Tensor Comprehensions was path-finding research and is no longer maintained. The git repo is frozen (archived) as a research artifact. Halide is still quite active, and was used in products at Adobe and Google circa 2016-2017. Not sure about the current state of industry usage though.

I'm unfamiliar with the word "path-finding" in this context. From the usage, is it similar to groundbreaking?

Re: Flashlight: Fast and flexible machine learning in C++

#86

Post by v8dev123 »

Kranar wrote: post #63
stephc_int13 wrote: post #53

Given the very high computing requirements of Machine Learning, I've always be perplexed by the seemingly widespread and unquestionable preference of Python over native code (typically C/C++). I guess performance was considered less critical than clarity/flexibility. But it seems that people are discovering that complex code tend to be hard to read/modify no matter the language...

I work at a quant firm and we use Python because of how painful it is to build things in C++. Our framework is built in C++ but we then expose all of it to Python using pybind11 (amazing library). Most quants do not want to learn complex build systems that have quirky behavior on different platforms, wait for very long compile times when making small changes to the code, dense and incomprehensible error messages, and…

Hmm

Which C++ version you guys are using?

Re: Flashlight: Fast and flexible machine learning in C++

#87

Post by lunixbochs »

whimsicalism wrote: post #23
dwrodri wrote: post #21

This seems really cool, but I don't get why they would pour work into this while simultaneously building a C++ front-end for PyTorch[1]. Per the blog post, both frameworks have the goal of empowering ML researchers to iterate on ML models in such a way that it becomes easier to reason about performance than it would be talking to a bunch of dynamically linked object files behind an interpreter. Facebook is a huge com…

Flashlight is much lower level and gives more fine-grained performance control. For instance, I don't think there is really any way to do real-time speech recognition that is fast with PyTorch because of how it is architected. From my understanding, Tensor Comprehensions and Halide are both very tentative research projects. > not having it's direction beholden to the whims of Alphabet (see: Swift for Tensorflow). I d…

I ship an app (Talon) which runs many kinds of wav2letter ASR models on consumer CPUs. It has real-time inference pipelines for both pytorch and flashlight. (I wrote the pytorch code). Performance of both inference pipelines is fine and comparable between the frameworks for me. I'm not sure what you're talking about wrt speech recognition performance.

Maybe for training? But there aren't that many cpu bound components there, and you can write those in native code.

Re: Flashlight: Fast and flexible machine learning in C++

#88

Post by whimsicalism »

lunixbochs wrote: post #87
whimsicalism wrote: post #23

Earlier quoted context omitted.

Flashlight is much lower level and gives more fine-grained performance control. For instance, I don't think there is really any way to do real-time speech recognition that is fast with PyTorch because of how it is architected. From my understanding, Tensor Comprehensions and Halide are both very tentative research projects. > not having it's direction beholden to the whims of Alphabet (see: Swift for Tensorflow). I d…

I ship an app (Talon) which runs many kinds of wav2letter ASR models on consumer CPUs. It has real-time inference pipelines for both pytorch and flashlight. (I wrote the pytorch code). Performance of both inference pipelines is fine and comparable between the frameworks for me. I'm not sure what you're talking about wrt speech recognition performance. Maybe for training? But there aren't that many cpu bound component…

Fair enough, I've followed your stuff a bit and you definitely know more about Flashlight than I do.

What would you say the motivation is for yet another ML framework, but in C++ this time?

Re: Flashlight: Fast and flexible machine learning in C++

#89

Post by dwrodri »

brobdingnagians wrote: post #76
dwrodri wrote: post #21

This seems really cool, but I don't get why they would pour work into this while simultaneously building a C++ front-end for PyTorch[1]. Per the blog post, both frameworks have the goal of empowering ML researchers to iterate on ML models in such a way that it becomes easier to reason about performance than it would be talking to a bunch of dynamically linked object files behind an interpreter. Facebook is a huge com…

The end really helped me with understanding their motivations: > Flashlight’s modular internals make it a powerful research framework for research frameworks. > We’re already using Flashlight at Facebook in our research focused on developing a fast speech recognition pipeline, a threaded and customizable train-time relabeling pipeline for iterative pseudo-labeling, and a differentiable beam search decoder. > Our ongo…

I'm a bit late to the reply, but I wanted to thank you for the succinct reponse. This is the exact sort of thing I was looking for when I made my comment.

Re: Flashlight: Fast and flexible machine learning in C++

#90

Post by Kranar »

v8dev123 wrote: post #86
Kranar wrote: post #63

Earlier quoted context omitted.

I work at a quant firm and we use Python because of how painful it is to build things in C++. Our framework is built in C++ but we then expose all of it to Python using pybind11 (amazing library). Most quants do not want to learn complex build systems that have quirky behavior on different platforms, wait for very long compile times when making small changes to the code, dense and incomprehensible error messages, and…

Hmm Which C++ version you guys are using?

MSVC 2019, g++ 10, clang 11.
Post reply on HN