Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thoughts on porting NumPy to PyPy

technicaldiscovery.blogspot.com

Post reply on HN
Print view View original

21–30 of 34 posts

Re: Thoughts on porting NumPy to PyPy

#21

Post by wladimir »

wesm wrote: post #19
fuzzyman wrote: post #17

Earlier quoted context omitted.

I don't really know what you mean by supporting numpy on pypy "excludes" scipy (other than in the short term). If you mean you assumed that the pypy team would port all of scipy as well as numpy, that seems like an unrealistic expectation for an initial port! An initial port seems like the only way forward from the point of view of the pypy team. I think it is unrealistic to expect the pypy team to take on the work o…

Well, the big problem to solve with SciPy is the fact that there is more, respectively, of C, C++, and Fortran in the SciPy codebase than Python ( http://www.ohloh.net/p/scipy ). Part of why Python has succeeded in scientific computing is integration of legacy codebases (via f2py or C extensions or...). There are probably man-years of work involved in devising a solution-- which by the time it's complete may be basic…

Have you looked at Theano ( http://deeplearning.net/software/theano/ ) ? It is a Python-based JIT for GPUs. Using Python you can build the computation pipeline symbolically, and the formulas are automatically converted to GPU code and scheduled as deemed fit (this can be extended to multiple GPUs, and could theoretically scale to an even higher level).

I think this is a promising idea for the future of array-oriented computing, as it can make use of one more level of parallelism / scaling than the current Numpy paradigm, which is limited to one operation at a time and the user providing the ordering of operations.

Re: Thoughts on porting NumPy to PyPy

#22

Post by another »

scott_s wrote: post #18
wesm wrote: post #8

Earlier quoted context omitted.

If you spent a day doing some serious linear algebra in Python you might change your tune. * (multiplication) between NumPy arrays by default does element-wise multiplication (potentially with broadcasting), which is the desired default behavior. In R, for example, you can define custom infix operators so that a * b is elementwise but a %* % b is matrix multiplication. If you write down a complicated linear algebra e…

I see your point, and it is why I tend to like operator overloading in languages. However, Python is a general purpose language, and scientific computing is a single domain. It is used in many different domains. Consider that there are many changes that individual communities would like, and if Python granted all of those requests, the language would be a mess. That's the challenge in designing a general purpose prog…

If we define "general purpose" as "equally mediocre in all domains", then what you say makes sense.

If we define "general purpose" as "works well in as many domains as possible", then I'd argue that the correct response to an easily-remedied weakness in an important domain should be to address that weakness.

As you imply, the challenge here is to decide which requests to grant and which to deny, but the mere fact that a request is "domain-specific" (ignoring, for the moment, the questionable idea that _linear algebra_ is domain-specific) should not be enough to rule it out.

(As an aside, scientific / numeric Python is, I believe, one of its two or three most important application areas, and perhaps the most important historical factor in its success: that community has been championing Python since the days when people were using Perl for the web, or for anything else. Notice, for example, that Travis mentions working on SciPy in 1999---and the original "Numeric" package was written in 1995.)

Re: Thoughts on porting NumPy to PyPy

#23

Post by IanOzsvald »

fuzzyman wrote: post #17
IanOzsvald wrote: post #15

Earlier quoted context omitted.

I spoke with Travis on Friday at Enthought and he opened my eyes to some of the possible problems of the pypy-numpy effort (which I've started discussing: http://mail.python.org/pipermail/pypy-dev/2011-October/00860... ). I hadn't realised that the port might exclude use of the rest of SciPy, this strikes me as a massive missed opportunity if it comes to pass. Perhaps a few other knowledgeable folk could post in the…

I don't really know what you mean by supporting numpy on pypy "excludes" scipy (other than in the short term). If you mean you assumed that the pypy team would port all of scipy as well as numpy, that seems like an unrealistic expectation for an initial port! An initial port seems like the only way forward from the point of view of the pypy team. I think it is unrealistic to expect the pypy team to take on the work o…

'excludes' scipy in that the numpy C API won't be available so there's a massive hill to climb (as best I understand it) to support most of scipy. Re. getting the pypy team to do the whole port - agree that'd be entirely unrealistic! Re. initial port - since there is more than one way forwards and I want to financially support the project (having pledged £600 earlier), I'd like to understand the risks and benefits in the options.

Re: Thoughts on porting NumPy to PyPy

#24

Post by eastwest »

wesm wrote: post #8
eastwest wrote: post #6

I just don't see adding a special matrix infix operator to Python happening. It is too specialized, and yet matrix is not special enough that it must have its own operator. Isn't this what operator overloading is for? I suspect the real problem is NumPy's type system implementation, since data-types are not visible as different Python types.

If you spent a day doing some serious linear algebra in Python you might change your tune. * (multiplication) between NumPy arrays by default does element-wise multiplication (potentially with broadcasting), which is the desired default behavior. In R, for example, you can define custom infix operators so that a * b is elementwise but a %* % b is matrix multiplication. If you write down a complicated linear algebra e…

I actually agree with you. I much prefer operator overloading than unwieldy chained function calls. But it might be tough to convince the larger Python community to add it since it is only useful to a small part of the community.

The last comment about dtypes is that the size (shape in NumPy) should be part of the type; certainly element-type should be. A 1-D vector of double should not be the same Python type as 3-D array of characters or a 500x600 matrix. This creates havoc in a dynamic language. I once spent two days tracking down a bug caused by "*" multiplying two arrays instead of two matrices when I started using NumPy. Perhaps size is too much for a dynamic language to be part of type, but surely dimension and element-type should be reflected in Python type. It absolutely does not help that the documentation is littered with type-objects and object-objects; I appreciate this is how C implementations are, but for a beginner NumPy users, it is more than a little confusing.

I have a lot of respect for people who designed and built NumPy; for multi-dimensional arrays I don't see a better approach; but too much of influence of C implementation details seep into Python interface, types and operator overloading are only the beginning of the problems. I am having second thought about the suitability of dynamic languages for large scale, high-performance computing. Type annotation could help a lot but I see there is exactly zero interest in that for NumPy.

Re: Thoughts on porting NumPy to PyPy

#25

Post by cdavid »

vgnet wrote: post #20
wesm wrote: post #19

Earlier quoted context omitted.

Well, the big problem to solve with SciPy is the fact that there is more, respectively, of C, C++, and Fortran in the SciPy codebase than Python ( http://www.ohloh.net/p/scipy ). Part of why Python has succeeded in scientific computing is integration of legacy codebases (via f2py or C extensions or...). There are probably man-years of work involved in devising a solution-- which by the time it's complete may be basic…

AFAIUI, what you're missing is that PyPy can (and often does) interface directly to C libraries, from RPython. So the prospect of re-implementing those specialized codebases isn't a real issue: only the CPython-API based wrappers would need re-implementing. I believe those are a small part of the total code you mention.

But those packages do not just depend on C libraries. They also depend on the numpy C API. If emulating the C API of CPython is too much non-fun work, I would expect the same to be true for numpy C API.

Re: Thoughts on porting NumPy to PyPy

#26

Post by cdavid »

fuzzyman wrote: post #16
philipn wrote: post #2

I love it when someone writes an articulate, non-inflamatory blog post with reasonable suggestions on how to improve things. It's sad how rare this is around here these days.

I missed the specific suggestions on how the pypy guys could improve things - other than "do a lot of work on numpy itself as well so that it better supports multiple implementations" (paraphrase obviously). That really needs to be the side of the equation that the numpy team take hold of.

I don't think Travis is suggesting to the pypy people to do all the work, but rather contribute to the discussion to help the numpy community as a whole for a way forward that makes it easier for both pypy and numpy own needs.

Re: Thoughts on porting NumPy to PyPy

#27

Post by kragen »

fijal wrote: post #13
wesm wrote: post #3

I couldn't agree more. This is related to another recent HN posting ( http://news.ycombinator.com/item?id=3104598 ) and discussions at PyCodeConf in Miami. There's a general feeling (which I've experienced in many of my interactions with non-scientific Python programmers) that the folks working on pure Python don't really "get" the scientific Python community. We would all understand each other better if the PyPy fol…

Note that as PyPy we deliberately don't take part in the language design discussions. We won't implement the array infix operator precisely because other people make those decisions.

Why not? Don't you think the language design discussion would be improved by your knowledge?

Re: Thoughts on porting NumPy to PyPy

#28

Post by kragen »

scott_s wrote: post #18
wesm wrote: post #8

Earlier quoted context omitted.

If you spent a day doing some serious linear algebra in Python you might change your tune. * (multiplication) between NumPy arrays by default does element-wise multiplication (potentially with broadcasting), which is the desired default behavior. In R, for example, you can define custom infix operators so that a * b is elementwise but a %* % b is matrix multiplication. If you write down a complicated linear algebra e…

I see your point, and it is why I tend to like operator overloading in languages. However, Python is a general purpose language, and scientific computing is a single domain. It is used in many different domains. Consider that there are many changes that individual communities would like, and if Python granted all of those requests, the language would be a mess. That's the challenge in designing a general purpose prog…

> Python is a general purpose language, and scientific computing is a single domain.

I think this is shortsighted to the point of ignorance. "Scientific computing" here really means "performance-critical numerical computation on regular arrays". Basically all of the new things that people are doing with computers in the last five years and the next five years — machine learning and other statistics, software-defined radio, audio synthesis, real-time video processing, cool visual effects, speech recognition, machine vision, and 3-D rendering, and arguably Bitcoin — consist largely of performance-critical numerical computation on regular arrays. It's what GPUs are for. Five years ago, Numeric or NumPy was probably the best way to do that for a wide range of things, although a lot of people still use Matlab instead, and R deserves at least a mention. Today it's not clear. Five years from now there will be something much better than current NumPy, and it could be a better version of NumPy or it could be R or Matlab or Octave or something.

In short, "scientific computing" is not a single domain, but a set of capabilities increasingly important in many different domains.

Re: Thoughts on porting NumPy to PyPy

#29

Post by kragen »

mcherm wrote: post #9
eastwest wrote: post #6

I just don't see adding a special matrix infix operator to Python happening. It is too specialized, and yet matrix is not special enough that it must have its own operator. Isn't this what operator overloading is for? I suspect the real problem is NumPy's type system implementation, since data-types are not visible as different Python types.

I'm just wondering how much you were hurt by the ellipsis built-in. After all, the ellipsis built-in was (at least for many years) of absolutely no use to anyone other than those using certain libraries for scientific computing. The rest of us had to put up with the pain of having yet one more line in the documentation. Frankly, for me it didn't hurt much. As a general rule, "Special cases aren't special enough to br…

Perhaps also see my comment nearby about how "scientific computing" isn't really a narrow domain any more. It could add ammunition to your argument.

Re: Thoughts on porting NumPy to PyPy

#30

Post by hogu »

scott_s wrote: post #18
wesm wrote: post #8

Earlier quoted context omitted.

If you spent a day doing some serious linear algebra in Python you might change your tune. * (multiplication) between NumPy arrays by default does element-wise multiplication (potentially with broadcasting), which is the desired default behavior. In R, for example, you can define custom infix operators so that a * b is elementwise but a %* % b is matrix multiplication. If you write down a complicated linear algebra e…

I see your point, and it is why I tend to like operator overloading in languages. However, Python is a general purpose language, and scientific computing is a single domain. It is used in many different domains. Consider that there are many changes that individual communities would like, and if Python granted all of those requests, the language would be a mess. That's the challenge in designing a general purpose prog…

Matrix multiplication isn't specific to scientific computing, it's a pretty common and large part of mathematics, and math is pretty general purpose.
Post reply on HN