Top Deep Learning Projects • Hacker News Board

Re: Top Deep Learning Projects

#21

Post by redcalx » Thu, Aug 11, 2016, 11:25 AM UTC

cedricblack wrote: post #13

nl wrote: post #7

A few comments on some of these projects: Keras is pretty much the best way to do almost anything these days. If you are starting out learning, use ConvNet JS, but after that switch to Keras. TFLearn is really nice if you are already using Scikit. There's lot of frameworks on there: TensorFlow, Caffe, CNTK (that's a lot of stars for something no one outside MS uses!) Theano, Torch etc. But I think the sleeper there i…

I ported ConvNet JS to C# in order to really understand what's going on: https://github.com/cbovar/ConvNetSharp

Brilliant, I've been looking for projects like this. I'm currently working through a couple of RBM C# projects but will add this to my list of reference code.

Top tip: If you use the matrix and vector classes in math.net then you can optionally configure it to use optimised version of e.g. matrix multiplication, that map through to one of the providers, such as Intel Math Kernel Lib, OpenBLAS, and I think there's a CUDA provider too.

I tried the Intel MKL one and the dense matrix multiplication was about 60x faster than a plain C# version.

https://github.com/mathnet/mathnet-numerics

https://www.nuget.org/packages/MathNet.Numerics/

Re: Top Deep Learning Projects

#22

Post by nl » Thu, Aug 11, 2016, 11:46 AM UTC

denzil_correa wrote: post #20

nl wrote: post #17

Earlier quoted context omitted.

It's a well designed API for using deep neural networks rather than an API for doing optimized mathematical operations. Compare how you build some vaguely comparable models in Keras[1] and raw TensorFlow[2]. Keras uses TensorFlow (or Theano) underneath, so there is no performance penalty. It's like in Python machine learning, most people use Scikit instead of implementing things in numpy. [1] https://github.com/fchol…

Thanks! Your example to explain the differences makes me more leaning towards Keras. It feels like Scikit for Deep Learning.

Yes, that's a decent analogy. If you are already familiar with Scikit then TFLearn is worth looking at too.

Re: Top Deep Learning Projects

#23

Post by chestervonwinch » Thu, Aug 11, 2016, 12:14 PM UTC

denzil_correa wrote: post #10

nl wrote: post #7

A few comments on some of these projects: Keras is pretty much the best way to do almost anything these days. If you are starting out learning, use ConvNet JS, but after that switch to Keras. TFLearn is really nice if you are already using Scikit. There's lot of frameworks on there: TensorFlow, Caffe, CNTK (that's a lot of stars for something no one outside MS uses!) Theano, Torch etc. But I think the sleeper there i…

> Keras is pretty much the best way to do almost anything these days What makes Keras take the advantage?

Theano itself is more like a language, not a deep learning framework. There is no NeuralNetworkClassifier class, for example. Although, you could write a neural network library / framework using Theano, and it would have all the benefits of Theano (code compiled for the GPU, various common neural net ops available, etc.), which is what it looks like the Keras folks have done. I took a stab at this a while ago (1), but I didn't keep up on it. I haven't used Keras much, but it looks like it fills a much needed gap, which I'm glad for.

(1): https://github.com/notmatthancock/neural_network

Re: Top Deep Learning Projects

#24

Post by zk00006 » Thu, Aug 11, 2016, 1:00 PM UTC

I wander how many people still implement their own networks as opposed to use these prepared frameworks. Or do you guys stick to single framework or use some sort of mixture of tools?

Re: Top Deep Learning Projects

#25

Post by toxik » Thu, Aug 11, 2016, 1:15 PM UTC

zk00006 wrote: post #24

I wander how many people still implement their own networks as opposed to use these prepared frameworks. Or do you guys stick to single framework or use some sort of mixture of tools?

Implementing a basic neural network is a lot of work, to be honest. I tried in C, in fact even made it parallelizable. It was hell. Lots of hair-pulling. I thank the gods for Keras and keep my head down.

Re: Top Deep Learning Projects

#26

Post by gcr » Thu, Aug 11, 2016, 3:34 PM UTC

Torch is split across many different repositories. Many of the relevant issues occur in `torch/nn` or `torch/cunn`, for example. Only considering `torch/torch7` underestimates the popularity by quite a lot.

Re: Top Deep Learning Projects

#27

Post by MasterScrat » Thu, Aug 11, 2016, 4:13 PM UTC

What about Spark MLlib?

http://spark.apache.org/mllib/

Re: Top Deep Learning Projects

#28

Post by Joof » Thu, Aug 11, 2016, 4:29 PM UTC

toxik wrote: post #25

zk00006 wrote: post #24

I wander how many people still implement their own networks as opposed to use these prepared frameworks. Or do you guys stick to single framework or use some sort of mixture of tools?

Implementing a basic neural network is a lot of work, to be honest. I tried in C, in fact even made it parallelizable. It was hell. Lots of hair-pulling. I thank the gods for Keras and keep my head down.

Unsurprising. Normally, you at least have something like BLAS to do linear algebra work.

It takes maybe 15 minutes in Python / numpy for something basic like a 2-layer net, but backpropegation is a little annoying to get right. Thus, tensorflow (or theano or caffe or whatever).

Re: Top Deep Learning Projects

#29

Post by vonnik » Thu, Aug 11, 2016, 4:33 PM UTC

This is, frankly, a naive way to rank deep learning projects, because Github stars are cheap. Francois Chollet, the creator of Keras, comes out with a monthly ranking that takes other factors into account, such as forks, contributors and issues, all stronger signs of community and users. Here's his July update:

https://twitter.com/fchollet/status/753980621823750145

Most of these frameworks are Python-oriented: Keras, Theano, Caffe, TF, neon, Mxnet, etc. The space is saturated. If you look at deep-learning projects by language, then Torch stands out -- it has a Lua API. And Deeplearning4j is the most widely used in Java and Scala. You don't have to crowbar it into a Spark integration, like you do with TensforFlow. http://deeplearning4j.org/

MXnet is not talked about a lot, but it's growing fast. It was heavily used by Graphlab/Turi, recently bought by Apple, so the question is what will happen with it now.

Re: Top Deep Learning Projects

#30

Post by Eridrus » Thu, Aug 11, 2016, 5:45 PM UTC

nl wrote: post #7

A few comments on some of these projects: Keras is pretty much the best way to do almost anything these days. If you are starting out learning, use ConvNet JS, but after that switch to Keras. TFLearn is really nice if you are already using Scikit. There's lot of frameworks on there: TensorFlow, Caffe, CNTK (that's a lot of stars for something no one outside MS uses!) Theano, Torch etc. But I think the sleeper there i…

Keras support for recurrent models leaves a bit to be desired at this point, so it's great if it has what you want, but otherwise you have to start peeking under the hood, which may be harder than just learning the underlying framework.