Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Flashlight: Fast and flexible machine learning in C++

ai.facebook.com

Post reply on HN
Print view View original

71–80 of 96 posts

Re: Flashlight: Fast and flexible machine learning in C++

#71

Post by tastyminerals2 »

liuliu wrote: post #39
dwrodri wrote: post #25

Earlier quoted context omitted.

This is a very interesting claim. I find it credible because it stands to reason that projects like DeepSpeed[1] and TorchScript[2] wouldn't need to exist if inference performance of research PyTorch models was satisfactory for production, but often case it isn't. It appears as though Flashlight is built on ArrayFire. I haven't seen how gradients are managed in arrayfire-ml, but perhaps it is the case that the autogr…

Torch (Lua) predates TensorFlow and is Lecun's pet project for a few years at that point already. But Lua as a language is unpopular at a time. PyTorch would be a welcome addition then. But even if no PyTorch (nor Caffe2) in an alternative timeline, I would imagine FB would be stuck with Lua Torch for quite some time.

Don’t forget that Lua Torch backend is C while Pytorch is C++ mostly.

Re: Flashlight: Fast and flexible machine learning in C++

#72

Post by trhway »

dwrodri wrote: post #21

This seems really cool, but I don't get why they would pour work into this while simultaneously building a C++ front-end for PyTorch[1]. Per the blog post, both frameworks have the goal of empowering ML researchers to iterate on ML models in such a way that it becomes easier to reason about performance than it would be talking to a bunch of dynamically linked object files behind an interpreter. Facebook is a huge com…

> paying huge salaries and then putting them under a layer of management that thinks two different C++ ML frameworks with the same goals is a good idea.

FB makes $1.3M/employee while average pay is $120K, i.e. less than 10% - thus the employees are actually very cheap and FB can waste a lot of employee resource, like doing 10 ML frameworks, and even though ML related employees earn $1M+ it still wouldn't move the needle financially for FB.

Re: Flashlight: Fast and flexible machine learning in C++

#73

Post by dynamite-ready »

person_of_color wrote: post #60
codebolt wrote: post #43

Earlier quoted context omitted.

Absolutely. In 2010 I used FANN for my Msc thesis research, and found it pretty easy to make my own little training sim for stock price data on top of it. Haven't done any ML work since, but I always scratched my head over how Python became the most popular language for this domain.

stock price ml? why aren’t you rich?

Who says they're not...

Re: Flashlight: Fast and flexible machine learning in C++

#74

Post by dynamite-ready »

teruakohatu wrote: post #46
dynamite-ready wrote: post #32

Does anyone else think that C++ makes more sense for ML work than Python? I'd been thinking so for years. Both for deployment/performance and data wrangling purposes.

Take a look at Julia's Flux.jl. It has a really nice API and is quite intuitive to use at a low level, and has higher level components as well. Julia has a fast maturing data wrangling super-project (Queryverse).

I'm guessing you'd have (slightly) fewer deployment options with Julia though.

Re: Flashlight: Fast and flexible machine learning in C++

#75

Post by brobdingnagians »

This makes a lot of sense when done well. Researchers need to have full control and quickly change things with the least amount of cognitive overhead for comprehending the engine. For people who know C++ well, it is a minimalistic C++ library with exactly what you need and nothing else. No cross language concerns or bindings, just plain C++.

Using modern c++:

> Modern C++ also obviates the need for tasks like memory management while providing powerful tools for functional programming.

> Flashlight supports doing research in C++ with no need to adjust external fixtures or bindings and no need for adapters to do things like threading, memory mapping, or interoperating with low-level hardware.

Easy integration with other C++ libraries:

> Flashlight makes it trivial to build new low-level computational abstractions. You can cleanly integrate CUDA or OpenCL kernels, Halide AOT pipelines, or other custom C/C++ code with minimal effort.

Customizable with fast build times:

> And when you change Flashlight’s core components, it takes just seconds to rebuild the entire library and its training pipelines, thanks to its minimalist design and freedom from language bindings.

Re: Flashlight: Fast and flexible machine learning in C++

#76

Post by brobdingnagians »

dwrodri wrote: post #21

This seems really cool, but I don't get why they would pour work into this while simultaneously building a C++ front-end for PyTorch[1]. Per the blog post, both frameworks have the goal of empowering ML researchers to iterate on ML models in such a way that it becomes easier to reason about performance than it would be talking to a bunch of dynamically linked object files behind an interpreter. Facebook is a huge com…

The end really helped me with understanding their motivations:

> Flashlight’s modular internals make it a powerful research framework for research frameworks.

> We’re already using Flashlight at Facebook in our research focused on developing a fast speech recognition pipeline, a threaded and customizable train-time relabeling pipeline for iterative pseudo-labeling, and a differentiable beam search decoder.

> Our ongoing research is further accelerated by the ability to integrate external platform APIs for new hardware or compiler toolchains and achieve instant interoperability with the rest of Flashlight.

They found it easier and faster to make a framework optimized for their research than to try to iterate on a larger and more complex codebase. I think there is less cognitive overhead and less things to change when experimenting with a new idea, and since they've optimized for fast rebuild times, it is much faster to try out new things. They get native integration with the C++ ecosystem for free and since their team all know C++ well, it makes sense to just do it in plain C++.

Re: Flashlight: Fast and flexible machine learning in C++

#77

Post by pjmlp »

stephc_int13 wrote: post #56
andrew_v4 wrote: post #54

Earlier quoted context omitted.

Python (in common ML frameworks) is really just wrapping well optimized native code. The overhead is very minimal. The advantage here is about easy access to the internals. I don't know that any material speedup is expected just because its written in C++

I've worked at some point on a commercial game engine written in C++ at the core but with many Lua components and API, for convenience. The reasoning wat the same, all the heavy lifting done with fast native code, and everything written in Lua was mostly glue code without real performance impact. Turns out the engine was slow and difficult to maintain because of the many interfaces. They ditched it a few years later.…

That was the same reasoning why Unreal dropped UnrealScript.

https://www.gamasutra.com/view/news/213647/Epics_Tim_Sweeney...

However a couple of years later and Epic is bringing scripting back with Verse

https://twitter.com/saji8k/status/1339709691564179464?s=20

Because the problem is not the scripting, but how it is done.

I imagine that the engine you mention did not use any kind of compiler for Lua, e.g. LuaJIT, nor batch calls across the marshalling layer.

Same happens on the Web, JavaScript JIT can outperform WebAssembly if one keeps switching all the time between layers, instead of doing batch requests.

Re: Flashlight: Fast and flexible machine learning in C++

#78

Post by [deleted] »

flemishgun wrote: post #62

Very cool to see more C++-based machine learning efforts. The language still needs a good dataframe abstraction (maybe XFrame?), but with matrix algebra provided by Armadillo/Eigen and other long-time machine learning libraries like mlpack, Shogun, Shark (and if you want to include C, Darknet), personally I think the future is bright for machine learning in C++---especially for production and deployment applications.

[deleted]

Re: Flashlight: Fast and flexible machine learning in C++

#80

Post by jcelerier »

whimsicalism wrote: post #13
angry_octet wrote: post #9

Earlier quoted context omitted.

Yes, I could see the construction, I just can't think why they wouldn't anticipate the problems it could cause down the track.

What? Nobody in their right mind thinks "fleshlight" immediately upon hearing the word "flashlight" Is this some sort of performance art comment?

as a non-native english speaker I'll humbly admit that fleshlight was indeed the first word that popped into my mind when reading the title, because I never say flashlight but lampe-torche (while a "fleshlight" is an occasional fun present / recurrent meme for friends birthday or stuff like that)
Post reply on HN