Live data from Hacker News

Why Is NumPy Only Now Getting Funded?

numfocus.org

61–70 of 115 posts

Re: Why Is NumPy Only Now Getting Funded?

#61

Earlier quoted context omitted.

Octave’s “business model” is “give away for free an inferior clone of a commercial product”. Why is anyone surprised that that isn’t lucrative? Anyone who has the money to spend on Octave is likely to just pay for Matlab. The defining feature of Octave users is that they want to use Matlab’s library ecosystem but they’re too cheap or broke to pay for a copy of Matlab.

On the other hand, one of the other major data analysis platforms went the other way: R is a free implementation of S language. TIBCO still sells S-PLUS, but my impression is that R is much, much more popular. My impression is that Octave always lags a little behind Matlab, both in terms of features and performance, since the matlab language is essentially whatever The Mathworks says it is. R and S-PLUS both have a s…

How much money does R make? Are the main R contributors working as processional academics, do they have corporate jobs that pay them to work part-time on R, or are they funded full time by donations/grants, or is R just a hobby side project for them?

I’m not saying there’s anything wrong with Octave, or R, or GIMP, or ..., only that it’s difficult to make money on it.

This is especially true for Octave though, because someone who wants to do exploratory numerical computing and doesn’t have a specific need for Matlab’s library ecosystem is likely to use Python or Julia or R or ...

Re: Why Is NumPy Only Now Getting Funded?

#62
post #51

Original NumPy author here. I have a lot to say on this topic, given that it has literally consumed my life over the past 20 years. You can go here for some thoughts about some of this: http://technicaldiscovery.blogspot.com/ There are several articles there that relate but in particular http://technicaldiscovery.blogspot.com/2012/10/continuum-and... and http://technicaldiscovery.blogspot.com/2017/02/numfocus-past...…

Hey, Travis, one question: have you managed to avoid selling non-free software? Everyone seems to think that eventually the way to sell free software is to sell some secret sauce on the side. Is this something you have completely eschewed? I ask because for Octave this is a non-negotiable requirement. Partly because of the GPL, but mostly because we really do believe that the whole point of Octave is to get away from…

I believe it is true that you can make more money selling non-free software (some of the profits from which should always be used to make free software).

You can generate money only through free software, however. Here are three approaches I have found: 1) Consult on projects that use the free software and use some of the profits to support that free software, 2) Sell enterprise-grade support on the free software to big companies. This is much more than just help-desk and answer the phone. All commercial software comes with a big contract. You provide the same kind of contract just no license restrictions. Others can do the same and so you have to distinguish yourself by either having all or most of the experts on the software or just really good marketing. 3) Dual license using GPL3/AGPL for the free version and a commercial license and then aggressively go after people for GPL violations if they don't get the commercial license.

I don't like the relationships created by the third model --- your sales processes become aggressive and counter-service minded by definition. I don't see it scaling and really providing value.

The other two are really hard to impossible to get investors exited about and therefore you struggle to get the capital together you need to prime the customer pump.

There is a another general model with many corollaries where you basically "do something else" that uses the software as a critical part of the business and let the profits of that activity fund open-source development.

A lot of open-source these days is actually funded by this kind of activity (or from VC's hoping to profit from a promise of great wealth from this kind of activity).

Re: Why Is NumPy Only Now Getting Funded?

#63
post #39

Earlier quoted context omitted.

Google hires devs to develop similar libraries... to power their machine learning efforts. There is tons of numerical work in finance, too. I don't think the money is to be made by selling copies of a Matlab-like piece of software, but in the application of the tools.

This is spot on. There are plenty of jobs applying open source technologies, but far fewer building those libraries themselves. (I'm NumPy dev who works at Google on machine learning.)

Really? This is somewhat surprising to me. How has Grumpy impacted your work / numerical computing in Python at Google?

Re: Why Is NumPy Only Now Getting Funded?

#64
I appreciate this article being posted, and have the utmost respect for NumPy developers. The urgency and discrepancy between use of certain important open-source libraries, and their support, is bewildering sometimes.

As I was thinking about it, though, I'm not surprised NumPy hasn't been funded before. The reasons why say a lot about biases in memory.

It wasn't that long ago that the sorts of things NumPy does were seen as fairly niche, and in the domain of statistics or engineering. It's only with relatively recent interest in AI and DL that this has been seen as within the purview of Silicon Valley-comp sci-type business, as opposed to EE or something different. I still am kind of a little disoriented--the other day, looking through our university's course catalog, I realized that certain topics that would have been taught in the stats or psychology departments are now being seen as the territory of comp sci. Statisticians have written excoriations about being treated as if they don't exist, as comp sci blithely barrels forward, reinventing the wheel.

I'm not meaning to take sides with these issues, only pointing out that I think the world we live in was very different not so long ago. It might seem puzzling that NumPy hasn't had more funding, but I think that's in part because what it's most profitably used for now wasn't really seen as much more than academic science fiction not too long ago.

The other part of it too, is that until relatively recently, if you were to do numerical heavy lifting, you'd almost certainly be expected to do that in C/C++ or maybe Fortran. There's a tension in numerical computing, between the performance and expressiveness that's needed, and Python is on one end of that continuum, far from the end that is traditionally associated with complex numerical computing. Sure, you had things like MATLAB with Python in the same functional role, but those were largely seen as teaching tools, or something that engineers did for one-off projects, having learned to do that in school (I still think the use of python in ML derives from the use of Python as a teaching tool in uni).

I'm not trying to knock Python or NumPy or anything, just kind of trying to convey a different perspective, which is that I can remember a time not too long ago when the use of Python in numerics was seen as primarily didactic in nature, or for limited circumscribed applications.

FWIW, it seems to me Python is kind of on a path similar to what happened with javascript, which was treated as kind of an ancillary helper language on the web, until Google started pushing its limits. Then there was browser wars 2.0, and huge efforts put into javascript, and it became a main player in network computing. To me, there's a similar trend with Python: it really kind of existed as a language for prototyping and scripting tasks, and now finds itself in a different role than it has been used for traditionally, and projects in that area are getting an influx of money accordingly. What I see happening is (1) a blossoming diversity of numerical computing communities (Haskell, Python, Julia, Kotlin, Scala, Rust, Go, etc.), due to competition and variation in application scenarios and preferences, (2) a huge influx of resources being put into Python to make it more performant, or (3) people jumping ship from Python into one of those other platforms to get more bang-for-the buck [or (4) some combination of all of these.]

Re: Why Is NumPy Only Now Getting Funded?

#65
Is the idea that "foundational work" (in any field) can be done without "huge sacrifices" widely accepted?

It sounds a tad unrealistic to me, unsupported by history.

It's as if people want to have it both ways: Create innovative SW, but also don't take risks or make sacrifices.

Offer software "for free" (and belligerently oppose even something like GPL), but also get paid (preferably by the government, so the people actually footing the bill have no say in it) and be long-term sustainable.

What's next: get paid, but also don't pay income taxes? Give away project control, but also keep it? :)

All understandable desires, but a little schizophrenic.

Disclaimer: I am a big fan of open source and NumPy in particular. I mentor students and OSS newcomers, I even pay one full-time dev to work only on OSS. It's just that I try not to kid myself about where the time&money comes from and where it goes, and I try not to have random people pay for my hobbies.

Extremely relevant previous HN conversion on this topic:

https://news.ycombinator.com/item?id=14551355

Re: Why Is NumPy Only Now Getting Funded?

#66
Because developers generally don't know (or don't like) the outreach necessary to fundraise.

For example in numpy case - https://github.com/numpy/numpy.org/issues/9 That's a request in March 2017 to add a donation button to the website. Im not sure that 6 months back, if Numpy was legally structured to receive larger funding. I posted a similar comment (with many more replies) in the context of Octave and it's funding https://news.ycombinator.com/item?id=13604564

Tl;Dr Don't ask for donations - instead sell "gratitude-ware"

There are tons of people who WANT to support these projects, but you have to make it easy and accountable to do that. The best example that I usually give is Sidekiq.

@mperham is awesome that way "This is exactly why I disclosed my revenue: people won't know there's a successful path forward unless it's disclosed. I want more OSS developers to follow my lead and build a bright future for themselves based on their awesome software."

In fact, I believe there's a start-up to be done here. "Stripe Atlas for Open Source software"

Re: Why Is NumPy Only Now Getting Funded?

#67
I tried at work to get some money to the maintainer of iRAP, an RNA sequencing analysis pipeline we depend on heavily at the moment. But business sees this as wasted money, it's there for free, why not take it? Reading this, I think I'm going to double down on my efforts again. We get so much value out of a huge pile of FOSS software, we should be donating. Meanwhile we have spend piles of money on Matlab for years and we aren't even allowed to run Linux on our Laptops if we wanted to.

Re: Why Is NumPy Only Now Getting Funded?

#68
post #65

Is the idea that "foundational work" (in any field) can be done without "huge sacrifices" widely accepted? It sounds a tad unrealistic to me, unsupported by history. It's as if people want to have it both ways: Create innovative SW, but also don't take risks or make sacrifices. Offer software "for free" (and belligerently oppose even something like GPL), but also get paid (preferably by the government, so the people…

Who said people want to do "foundational work"?

The just want to write some specific software they enjoy writing together.

The question rather is: after said software has been proven valuable and widely used by companies, and when there remain some hard and/or non-enjoyable parts to be done to complete it/enhance it (specific drivers, documentation, advanced features, etc), why don't any of the users donate some money to help?

Re: Why Is NumPy Only Now Getting Funded?

#69

Earlier quoted context omitted.

Google hires devs to develop similar libraries... to power their machine learning efforts. There is tons of numerical work in finance, too. I don't think the money is to be made by selling copies of a Matlab-like piece of software, but in the application of the tools.

But you have to run through their interview gauntlet.

And their decimation.

Re: Why Is NumPy Only Now Getting Funded?

#70
post #67

I tried at work to get some money to the maintainer of iRAP, an RNA sequencing analysis pipeline we depend on heavily at the moment. But business sees this as wasted money, it's there for free, why not take it? Reading this, I think I'm going to double down on my efforts again. We get so much value out of a huge pile of FOSS software, we should be donating. Meanwhile we have spend piles of money on Matlab for years a…

I think someone on here recently suggested to a project that they provide an "enterprise subscription" or something like that with minor benefits to make it easier for businesses to justify (or maybe it made administration easier?) to "donate". You could suggest that to the maintainer, and then use that to convince business that you need that.
Post reply on HN