Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Flashlight: Fast and flexible machine learning in C++

ai.facebook.com

Post reply on HN
Print view View original

21–30 of 96 posts

Re: Flashlight: Fast and flexible machine learning in C++

#21

Post by dwrodri »

This seems really cool, but I don't get why they would pour work into this while simultaneously building a C++ front-end for PyTorch[1]. Per the blog post, both frameworks have the goal of empowering ML researchers to iterate on ML models in such a way that it becomes easier to reason about performance than it would be talking to a bunch of dynamically linked object files behind an interpreter.

Facebook is a huge company with lots of money, but production-ready ML frameworks are a HUGE undertaking. I don't get how tech companies can be simultaneously recruiting and doing interviews year-round, paying huge salaries and then putting them under a layer of management that thinks two different C++ ML frameworks with the same goals is a good idea.

Facebook's work on Tensor Comprehensions, Halide (not originally FB but they have contributed heavily), Glow, and PyTorch all contributed to the ML space by offering alternatives (with innovative UX/technical differences) to the Tensorflow ecosystem. Not all of these contributions had novelty, but I respect FB's choice to work on something for the sole purpose of not having it's direction beholden to the whims of Alphabet (see: Swift for Tensorflow).

I just don't see what this adds which FB isn't already working on within a different project. What am I missing?

1 = https://pytorch.org/cppdocs/

Re: Flashlight: Fast and flexible machine learning in C++

#22

Post by UncleOxidant »

dwrodri wrote: post #21

This seems really cool, but I don't get why they would pour work into this while simultaneously building a C++ front-end for PyTorch[1]. Per the blog post, both frameworks have the goal of empowering ML researchers to iterate on ML models in such a way that it becomes easier to reason about performance than it would be talking to a bunch of dynamically linked object files behind an interpreter. Facebook is a huge com…

I tend to agree. If they said they had cleaned up the C++ backend that PyTorch uses and are releasing it separately on it's own with some usability enhancements as Flashlight, I could see that - they wouldn't have to maintain some other new thing. But this is apparently a completely separate thing.

Re: Flashlight: Fast and flexible machine learning in C++

#23

Post by whimsicalism »

dwrodri wrote: post #21

This seems really cool, but I don't get why they would pour work into this while simultaneously building a C++ front-end for PyTorch[1]. Per the blog post, both frameworks have the goal of empowering ML researchers to iterate on ML models in such a way that it becomes easier to reason about performance than it would be talking to a bunch of dynamically linked object files behind an interpreter. Facebook is a huge com…

Flashlight is much lower level and gives more fine-grained performance control. For instance, I don't think there is really any way to do real-time speech recognition that is fast with PyTorch because of how it is architected.

From my understanding, Tensor Comprehensions and Halide are both very tentative research projects.

> not having it's direction beholden to the whims of Alphabet (see: Swift for Tensorflow).

I don't think this is an accurate recreation of the history that led to FB working on pytorch.

Re: Flashlight: Fast and flexible machine learning in C++

#24

Post by CharlesW »

andrew_v4 wrote: post #8
angry_octet wrote: post #2

The name has unfortunate verbal similarities with a well known sex toy, especially when different accents are considered. I am actually astounded they went ahead with this name.

Really? In North American english, a flashlight is a handheld light, there is no thought of the toy you're referring to when it is mentioned. That "light" got its name as a play on flashlight. It's funny to see that now become the primary connotation in some peoples minds.

> In North American english, a flashlight is a handheld light, there is no thought of the toy you're referring to when it is mentioned.

Sample size of 1, but this North American immediately thought "oh no, how unfortunate" on seeing the headline. (Admittedly, I do listen to more podcasts than the average person.)

Re: Flashlight: Fast and flexible machine learning in C++

#25

Post by dwrodri »

whimsicalism wrote: post #23
dwrodri wrote: post #21

This seems really cool, but I don't get why they would pour work into this while simultaneously building a C++ front-end for PyTorch[1]. Per the blog post, both frameworks have the goal of empowering ML researchers to iterate on ML models in such a way that it becomes easier to reason about performance than it would be talking to a bunch of dynamically linked object files behind an interpreter. Facebook is a huge com…

Flashlight is much lower level and gives more fine-grained performance control. For instance, I don't think there is really any way to do real-time speech recognition that is fast with PyTorch because of how it is architected. From my understanding, Tensor Comprehensions and Halide are both very tentative research projects. > not having it's direction beholden to the whims of Alphabet (see: Swift for Tensorflow). I d…

This is a very interesting claim. I find it credible because it stands to reason that projects like DeepSpeed[1] and TorchScript[2] wouldn't need to exist if inference performance of research PyTorch models was satisfactory for production, but often case it isn't.

It appears as though Flashlight is built on ArrayFire. I haven't seen how gradients are managed in arrayfire-ml, but perhaps it is the case that the autograd implementation in PyTorch was a bottleneck and this is a ground up approach.

Editing as I didn't address your second point. I can neither confirm or deny the motivations for creating Torch being related to FB's desire to depend on an Alphabet-managed codebase. I know there are lots of reasons why programmers prefer the UX of the PyTorch Python API (I do as well), and there are probably other reasons I can't recall off the top of my head. I am only saying that PyTorch is contributing to the ML ecosystem already by the sole virtue that it isn't a Google product.

[1] = https://github.com/microsoft/DeepSpeed [2] = https://pytorch.org/docs/stable/jit.html

Re: Flashlight: Fast and flexible machine learning in C++

#26

Post by cultofmetatron »

whimsicalism wrote: post #19
angry_octet wrote: post #6

Earlier quoted context omitted.

I'd just choose a name with a greater Hamming distance from sexual harassment.

Now that I think about it... Docker does sound a bit like Dick-er.

theres also "docking." a sex act between two men of which the details I'll let you refer to google at your own discretion.

Re: Flashlight: Fast and flexible machine learning in C++

#28

Post by whimsicalism »

cultofmetatron wrote: post #26
whimsicalism wrote: post #19

Earlier quoted context omitted.

Now that I think about it... Docker does sound a bit like Dick-er.

theres also "docking." a sex act between two men of which the details I'll let you refer to google at your own discretion.

We really are an innovative species!

Re: Flashlight: Fast and flexible machine learning in C++

#29

Post by eptcyka »

angry_octet wrote: post #6
whoopdedo wrote: post #3

Earlier quoted context omitted.

And yet millions of electric torches are sold in the United States each year without confusion. (Although not, let's be honest, without amusement.)

I'd just choose a name with a greater Hamming distance from sexual harassment.

The sex toy came after flashlight the portable light source.
Post reply on HN