Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Leaf: Machine learning framework in Rust

github.com

Post reply on HN
Print view View original

41–50 of 53 posts

Re: Leaf: Machine learning framework in Rust

#41

Post by jean- »

wall_words wrote: post #20

The performance graph is deceptive for two reasons: (1) Leaf with CuDNN v3 is a little slower than Torch with CuDNN v3, yet the bar for leaf is positioned to the left of the one for Torch, and (2) there's a bar for Leaf with CuDNN v4, but not for Torch. It's good to see alternatives to Torch, Theano, and TensorFlow, but it's important to be honest with the benchmarks so that people can make informed decisions about w…

> Leaf with CuDNN v3 is a little slower than Torch with CuDNN v3, yet the bar for leaf is positioned to the left of the one for Torch

I think that's because they're sorting by forward time rather than forward+backward. That would also explain why in the Alexnet benchmark Tensorflow (cuDNN v4) is to the left of Caffe (cuDNN v3) despite having a much taller bar overall.

Re: Leaf: Machine learning framework in Rust

#42

Post by YeGoblynQueenne »

kvb wrote: post #31
YeGoblynQueenne wrote: post #16

> super-human image recognition That's a bold claim. As far as I know there was one paper that reported a model beating human scores in a specific test (imagenet, I believe). Whether that translates to "superhuman" results in general is followed by a very big question mark. In general I really struggle to see how any algorithm that learns from examples, especially one that minimises a measure of error against further…

Ensembles of humans can outperform the average human, and in the same way an algorithm trained on data labeled by an ensemble of humans can outperform the average human.

Beating the average human does not make you "superhuman". Here's a quick proof: there exist mere humans with above-average performance who can outperform the "average human". Those people are human. Therefore, they're not superhuman.

Besides, I have no idea whether the people who tagged Imagenet are the "average human", nor whether an ensemble of them can outperform the "average human".

Also, I'm pretty sure that it doesn't necessarily follow that an algorithm trained by many X can outperform any X. Most humans are trained by an ensemble of humans and they don't necessarily outperform the "average human".

Mind you, I'm not saying I _know_ what "superhuman" is, but then again I'm not the one who claims to have created an example of it.

Re: Leaf: Machine learning framework in Rust

#43

Post by YeGoblynQueenne »

Houshalter wrote: post #25
YeGoblynQueenne wrote: post #16

> super-human image recognition That's a bold claim. As far as I know there was one paper that reported a model beating human scores in a specific test (imagenet, I believe). Whether that translates to "superhuman" results in general is followed by a very big question mark. In general I really struggle to see how any algorithm that learns from examples, especially one that minimises a measure of error against further…

First of all no one ever expected machines to beat humans at Imagenet. At least not this soon. It's an amazing accomplishment, because Imagenet is high resolution pictures of many different types of objects. Which is very different than tiny photos or pictures of digits. Second the examples were produced by scraping Flickr. Then mechanical turkers were asked to confirm if the object was in the image or not. There are…

> First of all no one ever expected machines to beat humans at Imagenet.

I'm not contesting the fact that it's surprising and overall a sign of progress. I'm contesting the claim that it demonstrates "superhuman" performance.

By analogy, a good student at a bad school is "superhuman" because he or she got a good mark in an exam that most other pupils _in that school_ failed. You gotta go a lot further than that before you put on the red cape.

Re: Leaf: Machine learning framework in Rust

#44

Post by YeGoblynQueenne »

benbou09 wrote: post #19
YeGoblynQueenne wrote: post #16

> super-human image recognition That's a bold claim. As far as I know there was one paper that reported a model beating human scores in a specific test (imagenet, I believe). Whether that translates to "superhuman" results in general is followed by a very big question mark. In general I really struggle to see how any algorithm that learns from examples, especially one that minimises a measure of error against further…

It is much faster than humans

That means nothing.

Also, here's an example where humans beat machines in image recognition:

http://www.pnas.org/content/113/10/2744.full

The task is the recognition of very small and blurry images. Several different models were used, including a very deep convnet.

Re: Leaf: Machine learning framework in Rust

#45

Post by YeGoblynQueenne »

tomp wrote: post #24
YeGoblynQueenne wrote: post #16

> super-human image recognition That's a bold claim. As far as I know there was one paper that reported a model beating human scores in a specific test (imagenet, I believe). Whether that translates to "superhuman" results in general is followed by a very big question mark. In general I really struggle to see how any algorithm that learns from examples, especially one that minimises a measure of error against further…

> how is it possible to learn superhuman performance in anything from examples of mere human performance at the same task? I don't believe in magic. Computers could be better at assigning probabilities to ambiguous examples. In particular, for an image that is very ambiguous for most humans, maybe a computer would assign 99% probability to it (hence it would be only a little bit ambiguous).

That's not how it works. Assigning a high probability to anything is trivial: just add 90% to any probability calculation. The important thing is how close your guess is to the right answer.

Re: Leaf: Machine learning framework in Rust

#46

Post by kingnothing »

danielvf wrote: post #40
kingnothing wrote: post #29

I'm completely new to ML and what real world applications it's suitable for. Are we at the point yet where you can train a computer to look at arbitrary images and count the number of people in it? What if it was the largely on the same background and only the number of people were changing -- for example, a camera shooting a queue of people to determine queue depth at a bus station.

In the scale of computer vision problems, the stationary camera case is relatively easy. It's not too hard to isolate moving objects from a background, it's not too hard to decide if an object is a person or not, and it's not too hard to keep track of an object once you've identified it. You would still have to handle overlapping people, scene illumination changes, etc, but these can be solved and have been done befo…

Excellent, thanks. I'll take a look at that and hack around!

Re: Leaf: Machine learning framework in Rust

#47

Post by eranation »

This is very cool! When I presented it to my CTO however, he said that he doesn't think this will gain traction from data scientists over Scala or Python, as Rust is even more complex than Scala (which is not the simplest language out there, even though I'm a big fan of both Scala and Rust and I know this might start a flame war)

Do you think Data Scientists can write their models directly using Leaf? do you think there will need to be a DSL that translates form the R / Python world to something you can run on Leaf to make it happen?

Re: Leaf: Machine learning framework in Rust

#48

Post by kibwen »

wall_words wrote: post #20

The performance graph is deceptive for two reasons: (1) Leaf with CuDNN v3 is a little slower than Torch with CuDNN v3, yet the bar for leaf is positioned to the left of the one for Torch, and (2) there's a bar for Leaf with CuDNN v4, but not for Torch. It's good to see alternatives to Torch, Theano, and TensorFlow, but it's important to be honest with the benchmarks so that people can make informed decisions about w…

The graph in the readme is outdated, you can see the version with Torch/CuDNN v4 here: http://autumnai.com/deep-learning-benchmarks

And I don't believe the first point counts as deceptive; the bars are ordered by Forward ms, not by the sum of Forward and Backward. In both CuDNN v3 and v4, Leaf is faster than Torch by that metric (25 vs 28 for v4, 31 vs 33 for v3).

Re: Leaf: Machine learning framework in Rust

#49

Post by kibwen »

eranation wrote: post #47

This is very cool! When I presented it to my CTO however, he said that he doesn't think this will gain traction from data scientists over Scala or Python, as Rust is even more complex than Scala (which is not the simplest language out there, even though I'm a big fan of both Scala and Rust and I know this might start a flame war) Do you think Data Scientists can write their models directly using Leaf? do you think th…

By what metric does your CTO consider Rust to be more complex than Scala? A lot of Scala's complexity has to do with interfacing nicely with Java, and Scala has a lot of implicit behavior and TIMTOWTDI-ness that Rust deliberately tries to avoid. Odersky has even said that he's hoping that he can remove many features from Scala in the future.

Re: Leaf: Machine learning framework in Rust

#50

Post by emcq »

eranation wrote: post #47

This is very cool! When I presented it to my CTO however, he said that he doesn't think this will gain traction from data scientists over Scala or Python, as Rust is even more complex than Scala (which is not the simplest language out there, even though I'm a big fan of both Scala and Rust and I know this might start a flame war) Do you think Data Scientists can write their models directly using Leaf? do you think th…

It has less to do with complexity and more go do with REPL/Jupyter notebook support. Rust is a compiled language and you won't get some of the ease of exploratory data analysis you do with something like ipython.

I can use something like pandas or autograd to experiment with new optimization functions in seconds. For these big NN models it takes hours to days to wait for your model to train so squeezing out more performance is worth a more complex language.

Post reply on HN