Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine Learning Library for C++

image.diku.dk

Post reply on HN
Print view View original

31–40 of 52 posts

Re: Machine Learning Library for C++

#31

Post by pmelendez »

tlarkworthy wrote: post #17
hackinthebochs wrote: post #3

This looks awesome. I've been itching to try out some ideas I have after having gone through Bishop's book, but I've been hesitant to write the algorithms from scratch. Now I'll have to decide between learning matlab or a library such as this.

You should definitely use a scriptable ML library. The process is very iterative and not suited to a compiled language like C++. I use skilearn alot, but also the matlab toolboxes or R are great. At its heart ML is alot of stats, so use something built for maths, not C++. It doesn't really make sense to break out C++ until you know exactly what algorithm and settings you need and your application is real time.

I am very agreed with this. I had even been thinking in making a DSL (Scheme based probably) oriented to ML instead of a library. I would found that more useful in the exploring phase.

Re: Machine Learning Library for C++

#32

Post by pmelendez »

pilooch wrote: post #28

Very interesting, but as a daily practitioner I am skeptical. First, this is a lot of code! As a C++ machine learning programmer, I am impressed as I know the pain (someone explains why, see comment https://news.ycombinator.com/item?id=5613797 ). Second, it contains a version of Blas and ublas as well as LBFGS and more, much more, coded from scratch as it seems. This seems too much for an ML library, and a lot to mai…

May I ask how do you become a C++ ML programmer? It is not an usual position but one I would definitely be interested in pursuing

Re: Machine Learning Library for C++

#33

Post by forrestthewoods »

kruhft wrote: post #20
jhartmann wrote: post #16

Earlier quoted context omitted.

Flame Suit On / Rant Mode On I actually really don't understand why anyone uses GPL for a library. I've been doing open source for a long long time, and love the GPL. I have code in the Linux kernel, and believe free software AND open source software are great solutions to very real problems in software engineering. Having open code just gives people more options, and I firmly believe it will win over time as far as…

Some people believe in software freedom more than you do.

Some people define "freedom" with 10 pages of restrictions of which the exact interpretation is continuously debated years after release. Other people consider that the opposite of freedom.

Re: Machine Learning Library for C++

#34

Post by pilooch »

pmelendez wrote: post #32
pilooch wrote: post #28

Very interesting, but as a daily practitioner I am skeptical. First, this is a lot of code! As a C++ machine learning programmer, I am impressed as I know the pain (someone explains why, see comment https://news.ycombinator.com/item?id=5613797 ). Second, it contains a version of Blas and ublas as well as LBFGS and more, much more, coded from scratch as it seems. This seems too much for an ML library, and a lot to mai…

May I ask how do you become a C++ ML programmer? It is not an usual position but one I would definitely be interested in pursuing

Sure, I started as a researcher 14 years ago, then drifted to what I thought was a sweet spot then, half-research / half-programmer. I say 'sweet spot' because many applications did require both the academic and the applied background at the time, so for the sake of thrilling applications, it was worth 'downgrading' to pure engineering work when needed. Now I believe the game has changed a bit, coursera and others are infusing the minds of engineers with highly technical knowledge far more rapidly than before.

Typically I am astonished at the number of implementations of deep learning techniques (Shark does include some, AFAIK).

My past experience is that I had write many AI algorithms myself because I could not find any suitable, free and/or open implementations (or other researchers would not share theirs ;) ).

Re: Machine Learning Library for C++

#35

Post by pmelendez »

pilooch wrote: post #34
pmelendez wrote: post #32

Earlier quoted context omitted.

May I ask how do you become a C++ ML programmer? It is not an usual position but one I would definitely be interested in pursuing

Sure, I started as a researcher 14 years ago, then drifted to what I thought was a sweet spot then, half-research / half-programmer. I say 'sweet spot' because many applications did require both the academic and the applied background at the time, so for the sake of thrilling applications, it was worth 'downgrading' to pure engineering work when needed. Now I believe the game has changed a bit, coursera and others ar…

Thanks! That's very inspiring :) I worked in a similar position for a short time long time ago but after that I hadn't been able to find something similar. Good thing to see that people can find those sweet spots! :D

It's very true what you said about all the implementations available now. Although, for most algorithms I tend to try to implement them myself as a learning experience, maybe the biggest exception is standard SVM since it is kinda tricky but even for that there are some online algorithms that are easy to implement.

Thanks for sharing your experience!

Re: Machine Learning Library for C++

#36

Post by Schneekatze »

pilooch wrote: post #28

Very interesting, but as a daily practitioner I am skeptical. First, this is a lot of code! As a C++ machine learning programmer, I am impressed as I know the pain (someone explains why, see comment https://news.ycombinator.com/item?id=5613797 ). Second, it contains a version of Blas and ublas as well as LBFGS and more, much more, coded from scratch as it seems. This seems too much for an ML library, and a lot to mai…

Hi, shark developer here.

First of all, we are glad that our library is discussed on this board! We are happy for every feedback we can get!

Regarding Performance: we try to get the key algorithms as fast as possible. And for the hardest parts we rely not on ublas, but use optional bindings to ATLAS. Speed was one of the key design criteria. We hope that we achieved that. Clearly this is no guarantee that every algorithm is fast, but in this case: just add a ticket!

Please bear in mind, that Shark is still in beta stage, and we are heavily developing it (I am right now working on the family of multi class SVMs). So for example parallelism using OpenMP is not fully integrated.

Re: Machine Learning Library for C++

#37

Post by jhartmann »

tlarkworthy wrote: post #25
bch wrote: post #21

Earlier quoted context omitted.

I'm in the Stanford/Coursera machine learning course right now, and something like this is nearly excactly what I've been looking for. As some others have said, GPLv3 is off-putting, but there is the LGPL mlpack lib ( http://www.mlpack.org/ ) (also C++). Personally, project-wise, the only way this could be improved is if the project were pure C, and a BSD, MIT, or similar license. Quite looking forward to checking th…

honestly you guys are crazy if you think shark is gonna help you learn machine learning. Its ideal for deployment of ML on things like embedded computer, robotics, games etc. where real time learning is required. Machine learning requires alot of experimentation and C++ is a terrible medium for that. There are loads of good machine learning libraries implemented for python and matlab. Pretty much every good paper in…

Actually you forget that performance when you need to train for days at a time is critical, if I use Octave/Matlab/R my current project might take months to train instead of weeks. All my ML code is high performance threaded C++. I recommend you use a good template linear algebra library like Eigen, you can do plenty of experimentation in C++. I find with a set of a few modern libraries and the required experience a C++ programmer is just as if not more efficient than a Python/R/Matlab programmer. It comes down to the skill of the programmer and the proper choice of libraries.

Re: Machine Learning Library for C++

#38

Post by jhartmann »

Schneekatze wrote: post #36
pilooch wrote: post #28

Very interesting, but as a daily practitioner I am skeptical. First, this is a lot of code! As a C++ machine learning programmer, I am impressed as I know the pain (someone explains why, see comment https://news.ycombinator.com/item?id=5613797 ). Second, it contains a version of Blas and ublas as well as LBFGS and more, much more, coded from scratch as it seems. This seems too much for an ML library, and a lot to mai…

Hi, shark developer here. First of all, we are glad that our library is discussed on this board! We are happy for every feedback we can get! Regarding Performance: we try to get the key algorithms as fast as possible. And for the hardest parts we rely not on ublas, but use optional bindings to ATLAS. Speed was one of the key design criteria. We hope that we achieved that. Clearly this is no guarantee that every algor…

You should consider using Eigen for linear algebra, I have personally found it much better performance wise than using bindings to ATLAS or other more standard linear algebra solutions. ML algorithms tend to be multistage (think about the update of weights with momentum in a Neural network for example), and the primitives available in ATLAS or a blas library are really too low level. Eigen since it generates code for a whole complicated expression can blow a standard linear algebra library out of the water for a certain class of problems. For others obviously the highly tuned vendor BLAS code would win, but I've seen huge speedups by using Eigen it fits well for complex ML operations.

Re: Machine Learning Library for C++

#39

Post by jhartmann »

Schneekatze wrote: post #36
pilooch wrote: post #28

Very interesting, but as a daily practitioner I am skeptical. First, this is a lot of code! As a C++ machine learning programmer, I am impressed as I know the pain (someone explains why, see comment https://news.ycombinator.com/item?id=5613797 ). Second, it contains a version of Blas and ublas as well as LBFGS and more, much more, coded from scratch as it seems. This seems too much for an ML library, and a lot to mai…

Hi, shark developer here. First of all, we are glad that our library is discussed on this board! We are happy for every feedback we can get! Regarding Performance: we try to get the key algorithms as fast as possible. And for the hardest parts we rely not on ublas, but use optional bindings to ATLAS. Speed was one of the key design criteria. We hope that we achieved that. Clearly this is no guarantee that every algor…

FYI, another bonus on Eigen is if you are using a Matrix-Matrix operation you will get multithreading with OpenMP for free. It isn't the most efficient way (you can do better threading yourself probably), but it gets most of the way there for stuff that will use GEMM operations.

Re: Machine Learning Library for C++

#40

Post by Schneekatze »

jhartmann wrote: post #38
Schneekatze wrote: post #36

Earlier quoted context omitted.

Hi, shark developer here. First of all, we are glad that our library is discussed on this board! We are happy for every feedback we can get! Regarding Performance: we try to get the key algorithms as fast as possible. And for the hardest parts we rely not on ublas, but use optional bindings to ATLAS. Speed was one of the key design criteria. We hope that we achieved that. Clearly this is no guarantee that every algor…

You should consider using Eigen for linear algebra, I have personally found it much better performance wise than using bindings to ATLAS or other more standard linear algebra solutions. ML algorithms tend to be multistage (think about the update of weights with momentum in a Neural network for example), and the primitives available in ATLAS or a blas library are really too low level. Eigen since it generates code for…

This is true, even though we would rather switch to Armadillo due to it's easier handling and better high-level behaviour.

Right now the linear algebra library we use -ublas- has the same behaviour as Eigen for BLAS1 type expressions. So it tries to generate optimal (non-SSE) code. Only for BLAS2 and 3 we fall back to the ATLAS-routines which has the same performance as Eigen on the interesting problem sizes.

//small edit In the end it is not so interesting whether the BLAS1-type expressions are fast as they make up < 1% of run time performance. The big chunks are the data processing inside the matrix-matrix multiplications of the Neural Networks and similar entities.

Post reply on HN