Live data from Hacker News

Array Programming with NumPy

nature.com

41–50 of 114 posts

Re: Array Programming with NumPy

#41

BLAS and LAPACK were originally written for FORTRAN, and FORTRAN is particularly suited to numerical computing and array programming. https://modelingguru.nasa.gov/docs/DOC-1762

Aren't BLAS and LAPACK just for the linear algebra bits? Numpy is so much more than linear algebra.

Re: Array Programming with NumPy

#42

BLAS and LAPACK were originally written for FORTRAN, and FORTRAN is particularly suited to numerical computing and array programming. https://modelingguru.nasa.gov/docs/DOC-1762

Aren't BLAS and LAPACK just for the linear algebra bits? Numpy is so much more than linear algebra.

Yes, there's a lot more to it than that.

For instance, the interface and array/matrix types make vector operations really natural and efficient in python.

Re: Array Programming with NumPy

#43
post #38
post #32

Earlier quoted context omitted.

Well that could have been said (and was said) about Numpy/Scipy when it started, "oh R has so many more packages, what numpy can do I can do in MATLAB ...", yet here we are.

You probably don't realize it, you are agreeing with me :-)

It depends on the definition of 'people'. There were many who adopted numpy much before the ecosystem had had time to catchup. But I would readily concede that Dr. Jones @national_lab didnt at that time, in fact he probably hasnt even now.

I do disagree strongly with the opinion that Numpy is no better than MATLAB :). MATLAB has adopted some Numpy features after Numpy came out (broadcasting for example) but Numpy offered some genuine and unique advantages, both technical (broadcasting, no need for a MEX compiler that I have to pay through my nose for, not restricted to weird naming conventions, nature of parameter passing, ...) and legal.

Re: Array Programming with NumPy

#44
post #29

Earlier quoted context omitted.

Theoretically Julia is better for scientific computing, the only issue is its package ecosystem isn't as mature as Python's. But it's growing incredibly fast and there have already been libraries available for a few years that would be really impractical to write and maintain at such a level in C++ for Python. I assume that for science at least Julia will catch on a lot.

> Theoretically Julia is better for scientific computing, the only issue is its package ecosystem isn't as mature as Python's. That's not a small issue. The ecosystem is probably the reason people choose NumPy over MATLAB, for example. NumPy is not inherently superior to MATLAB, and most academicians that adopted NumPy in the 2000's already had a MATLAB license, so cost was not a concern either.

I started using python and numpy/scipy back then because it was vastly easier to deploy on a server or supercomputer. The matlab compiler meanwhile is clunky and adds new bugs and additional steps. Julia doesn't really match python in this regard either.

For more pure research and prototyping things both can do, I still think matlab is better though I rarely use it. I just like the idea of being able to easily deploy the code later somehow. Kind of an entrepreneurial feature.

Re: Array Programming with NumPy

#45
post #29

Earlier quoted context omitted.

Theoretically Julia is better for scientific computing, the only issue is its package ecosystem isn't as mature as Python's. But it's growing incredibly fast and there have already been libraries available for a few years that would be really impractical to write and maintain at such a level in C++ for Python. I assume that for science at least Julia will catch on a lot.

> Theoretically Julia is better for scientific computing, the only issue is its package ecosystem isn't as mature as Python's. That's not a small issue. The ecosystem is probably the reason people choose NumPy over MATLAB, for example. NumPy is not inherently superior to MATLAB, and most academicians that adopted NumPy in the 2000's already had a MATLAB license, so cost was not a concern either.

I think Numpy made it easier for people to integrate with a ton of open source libraries. Since everything is proprietary and the users are in a few specific niches, Matlab can't be as versatile. Also, each of the Matlab add-ons are another expensive license people are reluctant to pay. Sure, there are a bunch of contributed libraries for specific tasks, but comparatively the community is pretty poor.

Re: Array Programming with NumPy

#46
post #28

Don't underestimate the impact this has on getting funding or even just tenure/etc recognition for working on numpy. I'm in industry these days, but coming from the academic side, it's _really_ hard to get recognized for building the underlying infrastructure that tons of people use. I've built and maintained libraries that are used in a ton of publications, but was always told my work was "utterly and completely use…

I recall a story where a friend was unable to publish a paper in which he wrote an alternative to a very commonly used commercial tool (that virtually everybody used) with roughly 10 times better performance. He open sourced it and all, it was extremely useful, but there was no new methodology, it was simply very well implemented.

At a talk of his it lead to a very heated discussion where an older professor accused him of wasting government money on such nonsense.

Re: Array Programming with NumPy

#47
post #28

Don't underestimate the impact this has on getting funding or even just tenure/etc recognition for working on numpy. I'm in industry these days, but coming from the academic side, it's _really_ hard to get recognized for building the underlying infrastructure that tons of people use. I've built and maintained libraries that are used in a ton of publications, but was always told my work was "utterly and completely use…

I recall a story where a friend was unable to publish a paper in which he wrote an alternative to a very commonly used commercial tool (that virtually everybody used) with roughly 10 times better performance. He open sourced it and all, it was extremely useful, but there was no new methodology, it was simply very well implemented. At a talk of his it lead to a very heated discussion where an older professor accused h…

Been there. A few years back I got a government scholarship for my PhD (which is still in progress, due to my follow up work). I basically built the foundation upon which to establish a new field for my university, and the region where I live. There are some professor who think that scholarship (and the little money it gave me) was wasted on my because I chose to build all of that from the ground up, instead of rushing through my PhD.

By the way, those of that opinion are all professors who wanted me on their labs, but I turned them down...

Re: Array Programming with NumPy

#48
post #29

Earlier quoted context omitted.

> Theoretically Julia is better for scientific computing, the only issue is its package ecosystem isn't as mature as Python's. That's not a small issue. The ecosystem is probably the reason people choose NumPy over MATLAB, for example. NumPy is not inherently superior to MATLAB, and most academicians that adopted NumPy in the 2000's already had a MATLAB license, so cost was not a concern either.

I started using python and numpy/scipy back then because it was vastly easier to deploy on a server or supercomputer. The matlab compiler meanwhile is clunky and adds new bugs and additional steps. Julia doesn't really match python in this regard either. For more pure research and prototyping things both can do, I still think matlab is better though I rarely use it. I just like the idea of being able to easily deploy…

In Julia, there is one package manager and it gets things right. https://docs.julialang.org/en/v1/stdlib/Pkg/ It's super nice to have no fragmentation when it comes to packaging. In Pkg, package states are immutable, always reproducible, and quick. Julia packages that have binary dependencies usually build them all for every platform using the binary builder infrastructure (https://github.com/JuliaPackaging/Yggdrasil). It makes cross platform installation robust and testable, and suuuper quick. Pkg really is the rolls royce of package managers.

Re: Array Programming with NumPy

#49
post #32
post #29

Earlier quoted context omitted.

> Theoretically Julia is better for scientific computing, the only issue is its package ecosystem isn't as mature as Python's. That's not a small issue. The ecosystem is probably the reason people choose NumPy over MATLAB, for example. NumPy is not inherently superior to MATLAB, and most academicians that adopted NumPy in the 2000's already had a MATLAB license, so cost was not a concern either.

Well that could have been said (and was said) about Numpy/Scipy when it started, "oh R has so many more packages, what numpy can do I can do in MATLAB ...", yet here we are.

I just don't like the BASIC derived syntax of Julia (and Ruby.) I wish there was a language that was typed, had python like classes, subroutines and lambdas but JS like anonymous functions that was fast like Julia or at least close to numpy in number crunching without needing a module written in C.

Re: Array Programming with NumPy

#50
post #28

Don't underestimate the impact this has on getting funding or even just tenure/etc recognition for working on numpy. I'm in industry these days, but coming from the academic side, it's _really_ hard to get recognized for building the underlying infrastructure that tons of people use. I've built and maintained libraries that are used in a ton of publications, but was always told my work was "utterly and completely use…

We usually manage a Somethinginformatics journal publication to detail infrastructure work. At least Zenodo etc. And annoy users of the software to cite (a doi isn’t much to add to a manpage or log output).

> for over a decade

Probably two even if NumPy came out in 2005

Post reply on HN