Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Is NumPy Only Now Getting Funded?

numfocus.org

Post reply on HN
Print view View original

31–40 of 115 posts

Re: Why Is NumPy Only Now Getting Funded?

#31

Post by webmaven »

danjoc wrote: post #20

>the entire scientific Python stack was essentially relying upon on the “free time” work of only about 30 people—and no one had funding! 30 people? I remember a time when a certain fruit company would enter a field, literally hire all 30 of those guys, and put them behind closed doors. Then in 2 years they'd dominate the field for the next decade. Are these guys turning down offers? Or is the fruit company that poorl…

The fruit company (and really all of AFGAM and much of the rest of the industry) is biased toward hiring recent grads or folks who have a clear record of steadily increasing responsibility, which disadvantages those who have followed a more... eclectic path, as well as those who have had major dislocations of some sort derail their career.

And almost by definition, "those 30 guys" building and maintaining a library with their volunteer labor in some "field" (that didn't exist as such until relatively recently) mostly won't have standard-looking career paths.

Sometimes there is a critical mass of those folks concentrated in a company, prompting an acquihire.

Re: Why Is NumPy Only Now Getting Funded?

#32

Post by santaclaus »

webmaven wrote: post #31
danjoc wrote: post #20

>the entire scientific Python stack was essentially relying upon on the “free time” work of only about 30 people—and no one had funding! 30 people? I remember a time when a certain fruit company would enter a field, literally hire all 30 of those guys, and put them behind closed doors. Then in 2 years they'd dominate the field for the next decade. Are these guys turning down offers? Or is the fruit company that poorl…

The fruit company (and really all of AFGAM and much of the rest of the industry) is biased toward hiring recent grads or folks who have a clear record of steadily increasing responsibility, which disadvantages those who have followed a more... eclectic path, as well as those who have had major dislocations of some sort derail their career. And almost by definition, "those 30 guys" building and maintaining a library w…

They seem more likely than others -- see the recent Homebrew author grousing about getting rejected by Google, only to land at Apple, for instance.

Re: Why Is NumPy Only Now Getting Funded?

#33

Post by ngoldbaum »

danjoc wrote: post #20

>the entire scientific Python stack was essentially relying upon on the “free time” work of only about 30 people—and no one had funding! 30 people? I remember a time when a certain fruit company would enter a field, literally hire all 30 of those guys, and put them behind closed doors. Then in 2 years they'd dominate the field for the next decade. Are these guys turning down offers? Or is the fruit company that poorl…

There isn't much money in writing libraries like NumPy. Why would a company hire a bunch of expensive devs to write some software to compete with Matlab? The market just isn't big enough to justify the cost.

Re: Why Is NumPy Only Now Getting Funded?

#34

Post by williamstein »

webmaven wrote: post #15
projectramo wrote: post #11

Q: Are we conflating two issues? Is there a difference between the "sole developer problem" and the "lack of funding" problem. I mean, even if a project finds funding, does it follow that it will attract more talented developers? One way to distinguish the two issues is to look at for-profit software. In the cases where there is one primary developer, do they find it easy to keep the software going when the person re…

> even if a project finds funding, does it follow that it will attract more talented developers? It depends on what the funding is used for. Outreach in various forms (better documentation, presenting at conferences, a nicer website, moderating community venues from mailing lists to chat channels, triaging and grooming issues and PRs, etc.) takes time and effort (and therefore needs to be funded) as well.

For SageMath, what funding we’ve got has been mostly used for organizing many “Sage Days” workshop coding sprints – we’ve had nearly 100 now over the last decade! They have made an absolutely tremendous difference in attracting the over 600 people who have contributed. But also key design decision, e.g., 100% test coverage, helped too. I clearly remember the moment – in a big discussion at a Sage Days (funded by the Clay Math Institute) – that Craig Citro argued for 100% doctest coverage, and we all decided to do it. Without funding, those Sage Days wouldn’t have happened, and the community would not have developed. I also remember a Sage Days that we did jointly with Numpy at Enthought, where discussions and work we did together grew into many important things later. Funding is ridicuously absolutely critical to growing certain types of open source projects. In another direction, a lot of our infrastructure (e.g., build, testing, etc.,) is hosted on a big computer bought with a grant.

Re: Why Is NumPy Only Now Getting Funded?

#35

Post by rjbwork »

hiram112 wrote: post #30
rjbwork wrote: post #3

We have this problem in the .NET world. Accord.NET is written by a brilliant academic and programmer. It's well written, and has a good API, but it is largely the effort of this one dude, with minor contributions from a smattering of other fellows. Again, it is great in general, but it has bugs and rough edges here or there, and a lot of people don't trust it for production. I wish there was a way for people to be pr…

.Net had always had a problem with open source because Microsoft implemented APIs and kept them closed source. Users ended up paying for it via licensing, MSDN, and Visual Studio. I often see a lot of really mice Java frameworks with a .Net clone which gets much less attention (Hibernate, some of the Spring stuff, etc). Whole I understand Microsoft is going their best to open up .Net, I'm hesitant to believe they'll…

I started programming C# professionally in 2013...the ecosystem has grown and been improved by leaps and bounds since then. I'm hopeful, personally.

I think one thing that keeps C# things from being implemented elsewhere is that there are just very different styles between even C# and Java. Some fancy libraries will do lots of reflectiony things, build syntax trees, and compile on the fly for performance, allowing complex configuration to be simplified and get fast code at runtime. There's also a big culture of LINQ/functional style, and fluent interfaces in libraries, etc. which doesn't seem to be present in something like the Java community (at least until streams), whereas more dynamic/weakly typed languages don't have, for obvious reasons, a culture of strong typing and letting the compiler do the work for you.

Then when you go to the hardcore functional languages like Haskell, ML, etc. you don't have a culture of OO much, not to mention they just aren't used much in production environments.

So, you see a lot of things from elsewhere ported and brought to C#/.NET, but not a lot of things moving from C# to other ecosystems.

Re: Why Is NumPy Only Now Getting Funded?

#36

Post by santaclaus »

ngoldbaum wrote: post #33
danjoc wrote: post #20

>the entire scientific Python stack was essentially relying upon on the “free time” work of only about 30 people—and no one had funding! 30 people? I remember a time when a certain fruit company would enter a field, literally hire all 30 of those guys, and put them behind closed doors. Then in 2 years they'd dominate the field for the next decade. Are these guys turning down offers? Or is the fruit company that poorl…

There isn't much money in writing libraries like NumPy. Why would a company hire a bunch of expensive devs to write some software to compete with Matlab? The market just isn't big enough to justify the cost.

Google hires devs to develop similar libraries... to power their machine learning efforts. There is tons of numerical work in finance, too. I don't think the money is to be made by selling copies of a Matlab-like piece of software, but in the application of the tools.

Re: Why Is NumPy Only Now Getting Funded?

#37

Post by mschaef »

JPKab wrote: post #27
mschaef wrote: post #21

Earlier quoted context omitted.

I don't know who you're talking about, but Orange Micro went out of business in 2004. ( https://en.wikipedia.org/wiki/Orange_Micro )

Not sure if you're being sarcastic, but he means Apple.

It was more an attempt at light humor than sarcasm. :-)

(Not to mention I thought it was hard to imagine anybody around here being unaware of the most capitalized company in the world. Aside from maybe Saudi Aramco.)

Re: Why Is NumPy Only Now Getting Funded?

#38

Post by mschaef »

danjoc wrote: post #28
JPKab wrote: post #27

Earlier quoted context omitted.

Not sure if you're being sarcastic, but he means Apple.

I think he knows. Mike has a 5 digit /. ID :)

Heh... I knew my Slashdot ID was pretty low (~32K), but had no idea I'd been around here that long. Now I feel old. :-)

Speaking of old, I used Reddit as my main news source for a while, and I remember /r/programming having a post celebrating 65,536 users. Now it's over 10 times that, and there are more than three subreddits.

Re: Why Is NumPy Only Now Getting Funded?

#39

Post by shoyer »

santaclaus wrote: post #36
ngoldbaum wrote: post #33

Earlier quoted context omitted.

There isn't much money in writing libraries like NumPy. Why would a company hire a bunch of expensive devs to write some software to compete with Matlab? The market just isn't big enough to justify the cost.

Google hires devs to develop similar libraries... to power their machine learning efforts. There is tons of numerical work in finance, too. I don't think the money is to be made by selling copies of a Matlab-like piece of software, but in the application of the tools.

This is spot on. There are plenty of jobs applying open source technologies, but far fewer building those libraries themselves.

(I'm NumPy dev who works at Google on machine learning.)

Re: Why Is NumPy Only Now Getting Funded?

#40

Post by kevin_thibedeau »

santaclaus wrote: post #36
ngoldbaum wrote: post #33

Earlier quoted context omitted.

There isn't much money in writing libraries like NumPy. Why would a company hire a bunch of expensive devs to write some software to compete with Matlab? The market just isn't big enough to justify the cost.

Google hires devs to develop similar libraries... to power their machine learning efforts. There is tons of numerical work in finance, too. I don't think the money is to be made by selling copies of a Matlab-like piece of software, but in the application of the tools.

But you have to run through their interview gauntlet.
Post reply on HN