Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Homogenization of scientific computing – Python is eating other languages’ lunch

r-bloggers.com

Post reply on HN
Print view View original

11–20 of 184 posts

Re: Homogenization of scientific computing – Python is eating other languages’ lunch

#11

Post by synparb »

StefanKarpinski wrote: post #8
zwieback wrote: post #5

Too bad the first part of the post title was edited out of the HN title. I think outside of scientific computing the picture is a little more nuanced.

Even inside of scientific computing, the picture is quite a bit more nuanced. This post is basically about a the author's personal migration to Python as a user of other people's scientific programming packages. In doing interviews with people inside of companies, there's fairly little actual use of Python for scientific computing – lots of Python for data preparation, but R and Matlab (not to mention Simulink) still…

Although I would say in academia I've seen a rapid expansion of Python as both a Matlab replacement and for doing non-HPC work, at least in the field I work in (computational chemistry and biomolecular simulation).

Re: Homogenization of scientific computing – Python is eating other languages’ lunch

#12

Post by hharrison »

StefanKarpinski wrote: post #8
zwieback wrote: post #5

Too bad the first part of the post title was edited out of the HN title. I think outside of scientific computing the picture is a little more nuanced.

Even inside of scientific computing, the picture is quite a bit more nuanced. This post is basically about a the author's personal migration to Python as a user of other people's scientific programming packages. In doing interviews with people inside of companies, there's fairly little actual use of Python for scientific computing – lots of Python for data preparation, but R and Matlab (not to mention Simulink) still…

Yeah, I'm a Python convert like the author, though coming mostly from Matlab rather than R, and everyone in my field reacts with surprise when I tell them I prefer Python. They're open-minded, and I'm hoping to convert a few myself, but I don't think the mass migration has happened yet.

Regarding your second comment- you're correct of course, but what makes this a "blind spot"? After all, if the user is writing code in Python, they're doing scientific computing in Python, regardless of what the Python library calls behind the scenes. In my experience, a lot of people doing scientific computing--particularly those more interested in the science than the computing--could care less about what's going on behind the curtain. Any moment they have to think about implementation is a moment not thinking about science and therefore a waste of time. So it's actually a benefit for an ecosystem to hide the underlying mechanics--calling it a "bizarre blind spot" seems to imply they're doing something wrong.

Re: Homogenization of scientific computing – Python is eating other languages’ lunch

#13

Post by plafl »

bsg75 wrote: post #4
mrfusion wrote: post #2

My only question mark from this is matplotlib. I tried it five or six years ago and it seemed clunky to use and install. And worst I couldn't seem to just throw up a plot, I recall there being a lot of settings required. And the plots didn't look good by default you had to fool with fonts, font sizes, etc. Does anyone know if it's improved a lot since then? Otherwise I'm not seeing how it could hold a candle to R's p…

R may still have an advantage when it comes to plotting simplicity. On the Python side, matplotlib is still a bit of a pain, but has improved. Also look at ggplot.py (alpha-ish?) and Bokeh from ContinuumIO

What is simpler than this?

  x = arange(0, 2*pi, 1e-2)
  plot(x, sin(x))

Re: Homogenization of scientific computing – Python is eating other languages’ lunch

#14

Post by hharrison »

mrfusion wrote: post #2

My only question mark from this is matplotlib. I tried it five or six years ago and it seemed clunky to use and install. And worst I couldn't seem to just throw up a plot, I recall there being a lot of settings required. And the plots didn't look good by default you had to fool with fonts, font sizes, etc. Does anyone know if it's improved a lot since then? Otherwise I'm not seeing how it could hold a candle to R's p…

Yep, ggplot is the one thing that I keep coming back to R for, plus the odd statistical model I can't find in statsmodels - which is rarer and rarer.

One of the many awesome features of IPython - the interactive python shell and notebook, is that you can call code blocks in R just by prefacing with %%R. So my plotting habits are usually first to try the python port of ggplot, and if that can't handle my situation I just jump into R without having to switch windows or do any complicated data transfer.

It's worth mentioning that matplotlib is designed to mimic Matlab's plotting API, so for people coming from Matlab there's very little change, plus there's all the benefits of the other plotting libraries others have mentioned.

Re: Homogenization of scientific computing – Python is eating other languages’ lunch

#15

Post by leephillips »

plafl wrote: post #13
bsg75 wrote: post #4

Earlier quoted context omitted.

R may still have an advantage when it comes to plotting simplicity. On the Python side, matplotlib is still a bit of a pain, but has improved. Also look at ggplot.py (alpha-ish?) and Bokeh from ContinuumIO

What is simpler than this? x = arange(0, 2*pi, 1e-2) plot(x, sin(x))

This:

  set xrange [0:2*pi]
  plot sin(x)
EDIT: Yes, this is gnuplot, not R.

Re: Homogenization of scientific computing – Python is eating other languages’ lunch

#18

Post by coldtea »

hharrison wrote: post #12
StefanKarpinski wrote: post #8

Earlier quoted context omitted.

Even inside of scientific computing, the picture is quite a bit more nuanced. This post is basically about a the author's personal migration to Python as a user of other people's scientific programming packages. In doing interviews with people inside of companies, there's fairly little actual use of Python for scientific computing – lots of Python for data preparation, but R and Matlab (not to mention Simulink) still…

Yeah, I'm a Python convert like the author, though coming mostly from Matlab rather than R, and everyone in my field reacts with surprise when I tell them I prefer Python. They're open-minded, and I'm hoping to convert a few myself, but I don't think the mass migration has happened yet. Regarding your second comment- you're correct of course, but what makes this a "blind spot"? After all, if the user is writing code…

>Regarding your second comment- you're correct of course, but what makes this a "blind spot"? After all, if the user is writing code in Python, they're doing scientific computing in Python, regardless of what the Python library calls behind the scenes.

What he means is, they can't expand the core primitives provided for them in Python itself, so they are constrained by what's given if they want performance. Unlike with, say, Julia.

Re: Homogenization of scientific computing – Python is eating other languages’ lunch

#19

Post by StefanKarpinski »

hharrison wrote: post #12
StefanKarpinski wrote: post #8

Earlier quoted context omitted.

Even inside of scientific computing, the picture is quite a bit more nuanced. This post is basically about a the author's personal migration to Python as a user of other people's scientific programming packages. In doing interviews with people inside of companies, there's fairly little actual use of Python for scientific computing – lots of Python for data preparation, but R and Matlab (not to mention Simulink) still…

Yeah, I'm a Python convert like the author, though coming mostly from Matlab rather than R, and everyone in my field reacts with surprise when I tell them I prefer Python. They're open-minded, and I'm hoping to convert a few myself, but I don't think the mass migration has happened yet. Regarding your second comment- you're correct of course, but what makes this a "blind spot"? After all, if the user is writing code…

I call it a "bizarre blind spot" because it seems like there's a silent consensus to never talk about this basic fact. It's a bit surreal attending SciPy and hearing all of these people talking about scientific computing in Python when almost every single person in the room spends the vast majority of their time and energy writing C code.

I disagree that the separation between implementation and user-land that's enforced by two-language designs like C/Python or C/R is socially beneficial:

1. If your high-level code doesn't perform fast enough (or isn't memory efficient enough), you're basically stuck. You either live with it or you have to port your code to a low-level language. Not impossible, but not ideal either.

2. When there are problems with some package, most users are not in a position to identify or fix those problems – because of the language boundary. If the implementation language and the user language are the same, anyone who encounters a problem can easily see what's wrong and fix it.

3. Basically a corollary of 2, but having the implementation language and user language be the same is great for "suckering" users into becoming developers. In other words, this isn't just a one-time benefit: as users use the high-level language, they automatically become more and more qualified to contribute to the ecosystem itself. It is crucial to understand that this does not happen in Python. You can use NumPy until the cows come home and you will be no more qualified to contribute to its internals than you were when you started.

These benefits aren't just hypothetical – this is what is actively happening with Julia, where almost all of its high-performance packages are written in Julia. In fact, I never realized just how important these social effects where until experiencing it first hand. The author of the article wrote:

> It turns out that the benefits of doing all of your development and analysis in one language are quite substantial.

It turns out that it is even more beneficial to not only do development and analysis, but also build libraries in one language. Of course, Julia has a lot of catching up to do, but it's hard to not see that the author's own logic implies that it eventually will catch up and surpass two-language systems for scientific computing.

Post reply on HN