Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

sebastianraschka.com

Post reply on HN
Print view View original

51–60 of 98 posts

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#51

Post by srean »

geomark wrote: post #28
rm999 wrote: post #7

I switched from mostly using R to Python about a year ago for gluing together my data pipeline (from data source all the way to production models and frontends/visualizations). It hasn't really impacted what I'm capable of doing or my productivity, except the standard extra googling that comes in the first couple years I use any language. The main reason I went for Python is purely practical: it's a language people o…

The part about sharing makes a lot of sense since Python use is so wide spread. The throwing-over-the-wall effect isn't a language specific issue, more of a work culture issue. Seems to me if you practice "literate programming" with R markdown you can greatly improve the sharing aspect and reduce the throw-it-over-the-wall issue.

Totally agree about it being a cultural thing, have first hand experience at some of the usual suspects. I have a far more cynical label for it: deliver your turd (typically formed in MATLAB) for someone else to polish. It is surprising how common this is in some places and groups. I mention groups because when the group moves from one place to another it brings that turd polishing culture along.

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#52

Post by srean »

leni536 wrote: post #42

And there is nothing wrong with C++. For linear algebra I use the armadillo library and it's really a nice wrapper around LAPACK and BLAS (and fast!). For some reason scientists are somewhat afraid of C++. For some reason you "have to" prototype in an "easier" language. Sure, you can't use C++ as a calculator as opposed to interpreted languages, but I see people being stuck with their computations at the prototyping…

I saw you mention Armadillo a few times, seems you are having a lot of fun with it. Another somewhat look-alike is Eigen, its pretty nice too. These are all modern takes on the original sin Blitz++ which is actually more full featured in what they can represent than Eigen, Armadillo, mublas (by Boost) and Blaze: it supports multidimensional arrays as opposed to just (2d) matrices. Where Blitz++ falls short of the competition is full use of SIMD instructions. However, G++ does a plenty good job of vectorization, but the king of the hill is still Intel's compiler.

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#53

Post by srean »

rasbt wrote: post #44
leni536 wrote: post #42

And there is nothing wrong with C++. For linear algebra I use the armadillo library and it's really a nice wrapper around LAPACK and BLAS (and fast!). For some reason scientists are somewhat afraid of C++. For some reason you "have to" prototype in an "easier" language. Sure, you can't use C++ as a calculator as opposed to interpreted languages, but I see people being stuck with their computations at the prototyping…

I agree with you. However, note that many people who are using Python for writing scientific code make use of C/C++ in one way or the other (aside from NumPy, SciPy, and Theano). For example, many people write the "most intensive" computations down in C/C++/Cython if they call those functions frequently -- Python becomes a wrapper. One example that pops into my mind is khmer ( https://github.com/dib-lab/khmer )

This is a common refrain: drop down to C, C++, Fortran for the computation intensive parts. It works, but only to a degree. The inefficiencies lie in the vectorization semantics of the host language(s) that leads to extra copies and extra levels of indirection. So this dual language mode of operation typically does not approach what one could have obtained had one disposed the baggage entirely, except for I/O. Usually in the quest for better speed, that is what remains, as one moves progressively larger portions of the application in the C, C++, Fortran part of the code. A reason I like Julia is that I can largely avoid this dual language annoyance, and enjoy the succinctness of pithy vectorized expressions using https://github.com/lindahua/Devectorize.jl

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#54

Post by microtonal »

srean wrote: post #52
leni536 wrote: post #42

And there is nothing wrong with C++. For linear algebra I use the armadillo library and it's really a nice wrapper around LAPACK and BLAS (and fast!). For some reason scientists are somewhat afraid of C++. For some reason you "have to" prototype in an "easier" language. Sure, you can't use C++ as a calculator as opposed to interpreted languages, but I see people being stuck with their computations at the prototyping…

I saw you mention Armadillo a few times, seems you are having a lot of fun with it. Another somewhat look-alike is Eigen, its pretty nice too. These are all modern takes on the original sin Blitz++ which is actually more full featured in what they can represent than Eigen, Armadillo, mublas (by Boost) and Blaze: it supports multidimensional arrays as opposed to just (2d) matrices. Where Blitz++ falls short of the com…

I am not sure about Intel's compiler, but OpenBLAS is pretty much on par with Intel MKL in most benchmarks I have seen recently, e.g. [1][2].

The thing that I like with Armadillo is that you can just pick the BLAS implementation that is the fastest on your platform and run with it, while still being high-level compared to BLAS.

[1] http://gcdart.blogspot.co.uk/2013/06/fast-matrix-multiply-an...

[2] https://github.com/tmolteno/necpp/issues/18

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#55

Post by srean »

microtonal wrote: post #54
srean wrote: post #52

Earlier quoted context omitted.

I saw you mention Armadillo a few times, seems you are having a lot of fun with it. Another somewhat look-alike is Eigen, its pretty nice too. These are all modern takes on the original sin Blitz++ which is actually more full featured in what they can represent than Eigen, Armadillo, mublas (by Boost) and Blaze: it supports multidimensional arrays as opposed to just (2d) matrices. Where Blitz++ falls short of the com…

I am not sure about Intel's compiler, but OpenBLAS is pretty much on par with Intel MKL in most benchmarks I have seen recently, e.g. [1][2]. The thing that I like with Armadillo is that you can just pick the BLAS implementation that is the fastest on your platform and run with it, while still being high-level compared to BLAS. [1] http://gcdart.blogspot.co.uk/2013/06/fast-matrix-multiply-an... [2] https://github.com…

> you can just pick the BLAS implementation that is the fastest on your platform and run with it

Just in case this is useful info, Eigen and Blaze works the same way, Blitz++ doesn't. Blaze seems to be doing the best in the benchmarks.

> OpenBLAS is pretty much on par with Intel MKL

I wont be surprised. I have seen ATLAS beat MKL on some BLAS functions on my runs. Sparse matrix multiply is particularly badly implemented in MKL (atleast was). One could beat their sparse multiply by using multiple instance of their own sparse matrix vector multiplies.

But the reason I mentioned ICC is that there is much more to SIMD than just linear algebraic operations. Non-linear operations that come up quite frequently in stats/ML are sin, cos, exp, log, tanh etc on vectors. GCC/G++ does a decent job now (the best part is that it emits info why it wasn't able to vectorize a particular loop. This lets you restructure the loops to ai the compiler), but ICC still rules.

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#56

Post by acconsta »

microtonal wrote: post #54
srean wrote: post #52

Earlier quoted context omitted.

I saw you mention Armadillo a few times, seems you are having a lot of fun with it. Another somewhat look-alike is Eigen, its pretty nice too. These are all modern takes on the original sin Blitz++ which is actually more full featured in what they can represent than Eigen, Armadillo, mublas (by Boost) and Blaze: it supports multidimensional arrays as opposed to just (2d) matrices. Where Blitz++ falls short of the com…

I am not sure about Intel's compiler, but OpenBLAS is pretty much on par with Intel MKL in most benchmarks I have seen recently, e.g. [1][2]. The thing that I like with Armadillo is that you can just pick the BLAS implementation that is the fastest on your platform and run with it, while still being high-level compared to BLAS. [1] http://gcdart.blogspot.co.uk/2013/06/fast-matrix-multiply-an... [2] https://github.com…

Not so surprising that OpenBLAS beat MLK on an AMD processor. Although it's definitely keeping up.

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#57

Post by IndianAstronaut »

Adam_O wrote: post #17

From the perspective of a student, most of the good online analytics/data analysis/stats courses use R, so it is hard to get away from it while learning the material. Once you get the base concepts down, switching to python shouldn't be hard. I think most people still prefer ggplot2 for visualization though. Whenever I use R I feel like a statistician, I can feel that 'cold rigor' emanating from the language. But in…

Jupyter is amazing. It is great to see workflow and makes it easy to graph and find trends in the data, as well as mistakes in a data flow.

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#58

Post by drauh »

rm999 wrote: post #7

I switched from mostly using R to Python about a year ago for gluing together my data pipeline (from data source all the way to production models and frontends/visualizations). It hasn't really impacted what I'm capable of doing or my productivity, except the standard extra googling that comes in the first couple years I use any language. The main reason I went for Python is purely practical: it's a language people o…

Yeah, there is a large community of Python users in scientific computing. It's great.

I like well-established languages with a large user base.

So, I was dismayed by Big Data Genomics' ADAM Project's choice of Scala, which has almost no uptake in the genomics/bioinformatics community.

They do it because they run over Spark. But Spark has an excellent Python binding.

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#59

Post by microtonal »

srean wrote: post #55
microtonal wrote: post #54

Earlier quoted context omitted.

I am not sure about Intel's compiler, but OpenBLAS is pretty much on par with Intel MKL in most benchmarks I have seen recently, e.g. [1][2]. The thing that I like with Armadillo is that you can just pick the BLAS implementation that is the fastest on your platform and run with it, while still being high-level compared to BLAS. [1] http://gcdart.blogspot.co.uk/2013/06/fast-matrix-multiply-an... [2] https://github.com…

> you can just pick the BLAS implementation that is the fastest on your platform and run with it Just in case this is useful info, Eigen and Blaze works the same way, Blitz++ doesn't. Blaze seems to be doing the best in the benchmarks. > OpenBLAS is pretty much on par with Intel MKL I wont be surprised. I have seen ATLAS beat MKL on some BLAS functions on my runs. Sparse matrix multiply is particularly badly implemen…

Just in case this is useful info, Eigen and Blaze works the same way, Blitz++ doesn't.

I was under the impression that Eigen only works with MKL?

Enables the use of external BLAS level 2 and 3 routines (currently works with Intel MKL only)

Source: http://eigen.tuxfamily.org/dox/TopicUsingIntelMKL.html

But the reason I mentioned ICC is that there is much more to SIMD than just linear algebraic operations.

Indeed. I compiled some stuff with ICC, but it did not give me an tangible improvement over the latest GCCs. But for those projects, linear operations were the vast majority.

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#60

Post by cmarschner »

leni536 wrote: post #42

And there is nothing wrong with C++. For linear algebra I use the armadillo library and it's really a nice wrapper around LAPACK and BLAS (and fast!). For some reason scientists are somewhat afraid of C++. For some reason you "have to" prototype in an "easier" language. Sure, you can't use C++ as a calculator as opposed to interpreted languages, but I see people being stuck with their computations at the prototyping…

For machine learning / neural nets in C++ there is also Caffe (for ConvNets) and CNTK (for almost any type of network).
Post reply on HN