Python, Machine Learning, and Language Wars. A Highly Subjective Point of View • Hacker News Board

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#71

Post by Lofkin » Tue, Aug 25, 2015, 11:11 AM UTC

srean wrote: post #53

rasbt wrote: post #44

Earlier quoted context omitted.

I agree with you. However, note that many people who are using Python for writing scientific code make use of C/C++ in one way or the other (aside from NumPy, SciPy, and Theano). For example, many people write the "most intensive" computations down in C/C++/Cython if they call those functions frequently -- Python becomes a wrapper. One example that pops into my mind is khmer ( https://github.com/dib-lab/khmer )

This is a common refrain: drop down to C, C++, Fortran for the computation intensive parts. It works, but only to a degree. The inefficiencies lie in the vectorization semantics of the host language(s) that leads to extra copies and extra levels of indirection. So this dual language mode of operation typically does not approach what one could have obtained had one disposed the baggage entirely, except for I/O. Usuall…

Numba for python also has a no copies vectorization mode.

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#72

Post by ninjin » Tue, Aug 25, 2015, 11:16 AM UTC

idunning wrote: post #5

As someone who almost exclusively uses Julia for their day-to-day work (and side projects), I think most of the author's thoughts about Julia are correct. I think the language is great, and using it makes my life better. There are some packages that are actually better than any of their equivalents in other languages, in my opinion. On the other hand, I've also got a higher tolerance for things not being perfect, I c…

As a fellow Julian, I agree. To reiterate what I said during my JuliaCon talk, I never truly understood what Rubyists meant when they said "optimised for happiness", Ruby never had that effect on me and I remained a happy Pythonista. However, Julia makes me feel this way and got me "hacking" again after having been in a bit of a slump.

That being said, we need to realise how young Julia actually is. Last Sunday marked six years since the first commit and the language has only been publicly available for three years. Julia is as old as Python was in 1994, or as old as Ruby was in 1998. Things are rough around the edges, but getting better. Things are missing in order for you to get productive quickly, but it is improving rapidly.

I do tell people around me that Julia is most likely the language that I love the most, I tell them about the features, my involvement, etc. But, when they say "That sounds awesome! Should I use it?", I hesitate and say, well, it depends. Are you already very productive in what you are currently using? Is speed an issue for you? If not, don't feel rushed, if you want to be a "pioneer" you certainly can be and we will welcome you. But consider your own situation before jumping ship, what do you need to be productive in your day-to-day job? However, keep an eye on Julia, because I am convinced that the cost of adaptation for you will continue to go down and if there is any language that has the chance to become "just right" for Machine Learning over the next couple of years, it is most likely going to be Julia.

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#73

Post by srean » Tue, Aug 25, 2015, 11:37 AM UTC

syllogism wrote: post #69

srean wrote: post #53

Earlier quoted context omitted.

This is a common refrain: drop down to C, C++, Fortran for the computation intensive parts. It works, but only to a degree. The inefficiencies lie in the vectorization semantics of the host language(s) that leads to extra copies and extra levels of indirection. So this dual language mode of operation typically does not approach what one could have obtained had one disposed the baggage entirely, except for I/O. Usuall…

You don't have to use the vectorization though. I prefer to work with Cython and just use pointers. You don't get to write vector + vector --- you have to actually write the loop --- but that's not such a big deal imo.

I am lazy and super prone to make one-off errors, so that to me is the _biggest_deal_ TM :)

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#74

Post by srean » Tue, Aug 25, 2015, 11:40 AM UTC

microtonal wrote: post #63

srean wrote: post #61

Earlier quoted context omitted.

Its a BLAS library standard API that you are linking to, so the vendor doesn't matter so much (in theory). In practice, the amount of annoyance that you have to go through so that it links correctly can vary a lot. Even MKL did not implement all of BLAS / Lapack and that would lead to some linking errors. Re GCC, the same here. ICC is mighty expensive.

I know that BLAS is a standard API, I switch between BLAS implementations fairly often :). I suspected that Eigen relied on some non-BLAS functions in MKL, since they explicitly state that only MKL works.

> I switch between BLAS implementations fairly often

... that makes us co-sufferers then, or brothers in masochism :) if you will

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#75

Post by nxb » Tue, Aug 25, 2015, 12:20 PM UTC

srean wrote: post #53

rasbt wrote: post #44

Earlier quoted context omitted.

I agree with you. However, note that many people who are using Python for writing scientific code make use of C/C++ in one way or the other (aside from NumPy, SciPy, and Theano). For example, many people write the "most intensive" computations down in C/C++/Cython if they call those functions frequently -- Python becomes a wrapper. One example that pops into my mind is khmer ( https://github.com/dib-lab/khmer )

This is a common refrain: drop down to C, C++, Fortran for the computation intensive parts. It works, but only to a degree. The inefficiencies lie in the vectorization semantics of the host language(s) that leads to extra copies and extra levels of indirection. So this dual language mode of operation typically does not approach what one could have obtained had one disposed the baggage entirely, except for I/O. Usuall…

The speed performance of C/C++ is terrible, compared to GPU speed, which is often 50x to 100x faster. The goal has shifted and continues to shift in that direction.

C/C++ is no longer best for speed, not even close.

Now, all that matters is which language has the libraries that make it easiest to get custom code onto the GPU. Python and Lua seem to be winning there, by far.

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#76

Post by leni536 » Tue, Aug 25, 2015, 12:31 PM UTC

nxb wrote: post #75

srean wrote: post #53

Earlier quoted context omitted.

This is a common refrain: drop down to C, C++, Fortran for the computation intensive parts. It works, but only to a degree. The inefficiencies lie in the vectorization semantics of the host language(s) that leads to extra copies and extra levels of indirection. So this dual language mode of operation typically does not approach what one could have obtained had one disposed the baggage entirely, except for I/O. Usuall…

The speed performance of C/C++ is terrible, compared to GPU speed, which is often 50x to 100x faster. The goal has shifted and continues to shift in that direction. C/C++ is no longer best for speed, not even close. Now, all that matters is which language has the libraries that make it easiest to get custom code onto the GPU. Python and Lua seem to be winning there, by far.

>Now, all that matters is which library makes it easiest to get custom code onto the GPU. Python and Lua seem to be winning there, by far.

This is interesting. How is it possible that python and lua have more efficient wrappers around GPU libraries? Also there are many GPU libraries for C/C++ too. Armadillo can use NVBLAS as a backend too. I'm not sure if I get your point of C/C++ being slow.

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#77

Post by nxb » Tue, Aug 25, 2015, 12:39 PM UTC

leni536 wrote: post #76

nxb wrote: post #75

Earlier quoted context omitted.

The speed performance of C/C++ is terrible, compared to GPU speed, which is often 50x to 100x faster. The goal has shifted and continues to shift in that direction. C/C++ is no longer best for speed, not even close. Now, all that matters is which language has the libraries that make it easiest to get custom code onto the GPU. Python and Lua seem to be winning there, by far.

>Now, all that matters is which library makes it easiest to get custom code onto the GPU. Python and Lua seem to be winning there, by far. This is interesting. How is it possible that python and lua have more efficient wrappers around GPU libraries? Also there are many GPU libraries for C/C++ too. Armadillo can use NVBLAS as a backend too. I'm not sure if I get your point of C/C++ being slow.

It's not about wrapping. The real power is in the cross-compilation of expressions and entire complex data pipelines, from a simple-as-possible high-level language into GPU language. That's the power at the core of, e.g. Theano.

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#78

Post by haddr » Tue, Aug 25, 2015, 1:10 PM UTC

RA_Fisher wrote: post #22

misiti3780 wrote: post #13

Octave/Matlab are "great" but good luck trying to integrate them into a production web application. Since you cant really do that - avoid using them unless you are fine with implementing the same algorithm twice. Matlab licenses cost money also, and the toolboxes cost additional money. R is useful because there are a lot of resources as it has been along for so long and is used by a large portion of the stats communi…

I'd like to kindly challenge the notion that you can't integrate R into a web application. I've started using R to power jobs that are used by a large web application. The R packages httr or RCurl make it pretty easy to make http requests -- (enabling me to send things to a web server to be consumed into a database and run by back-end code). It's also possible to prepare data in R and then send to a space like S3 wit…

I was once quite suprised that R was missing any kind of RESTful service package for exposing R functions as a REST service. What I was looking for was basically some way to invoke a couple of R functions from the web application. In the end I managed to do it with some JRI (part of rJava) bindings.

Maybe it's a good idea to implement such thing.

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#79

Post by haddr » Tue, Aug 25, 2015, 1:18 PM UTC

misiti3780 wrote: post #13

Octave/Matlab are "great" but good luck trying to integrate them into a production web application. Since you cant really do that - avoid using them unless you are fine with implementing the same algorithm twice. Matlab licenses cost money also, and the toolboxes cost additional money. R is useful because there are a lot of resources as it has been along for so long and is used by a large portion of the stats communi…

R in general is not a good stream language. It is much better for batch processing (using "vectorisation"). Anyway it doesn't mean you can't stream process data with R! Spark or Flink provide some means for grouping streams into windows that fit R very well.

But I agree that integrating R with something else is an unexplored terrain. It seems that R rather tends to be everything: from data acquisition to visualisation.

Re: Python, Machine Learning, and Language Wars. A Highly Subjective Point of View

#80

Post by srean » Tue, Aug 25, 2015, 2:03 PM UTC

nxb wrote: post #75

srean wrote: post #53

Earlier quoted context omitted.

This is a common refrain: drop down to C, C++, Fortran for the computation intensive parts. It works, but only to a degree. The inefficiencies lie in the vectorization semantics of the host language(s) that leads to extra copies and extra levels of indirection. So this dual language mode of operation typically does not approach what one could have obtained had one disposed the baggage entirely, except for I/O. Usuall…

The speed performance of C/C++ is terrible, compared to GPU speed, which is often 50x to 100x faster. The goal has shifted and continues to shift in that direction. C/C++ is no longer best for speed, not even close. Now, all that matters is which language has the libraries that make it easiest to get custom code onto the GPU. Python and Lua seem to be winning there, by far.

Did you just compare a language with a piece of hardware ? I hope you realize:

(i) how nonsensical such a comparison is

(ii) there are many algorithms for which GPU offers no speedup at all in fact the the data transfer can actually hurt. There are instances where using CPU's SIMD instructions makes more sense than GPU.