Live data from Hacker News

Ask HN:Why is Perl so dwarfed in data science by Python?

news.ycombinator.com

11–20 of 120 posts

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#11
PDL is just awesome, but I feel that Perl is a programmer's programming language. If you come from Linux, shell scripting, C/C++, etc, you'll probably handle Perl well.

However, it might be a bit too much if you are from other fields and just want to get your sums in order.

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#12
Python is not a good language for low-level numerics like FORTRAN, but it has good facilities to make foreign language libraries (written in C, FORTRAN, CUDA, etc.) that do high-level operations. There is not just the foreign function interface but also the operator overloading that makes it possible to write something like

   dataframe["A"] + dataframe["B"]
and get a Series. You can make things that look like numbers or arrays but actually do something different.

Both Python and Perl made bold transitions, Python from 2 to 3 and Perl from 5 to 6.

The Python transition was difficult but ultimately successful. On the other hand, Perl rolled the dice and lost. In some alternate universe it might have been the other way around.

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#13

tooling and network effects. perl doesn’t have the numerical chops to keep up, and if it started to fix that now, it has 20 years of headwind to fight through for probably marginal gains. Good luck.

Aren’t the numerical chops of Python actually Fortran?

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#15

Perl is expressive but the code can be hard to read. In general, Python is readable because it enforces the indentation and other language design choices. Python also got heavyweight libraries such as Numpy and Pandas which put it in the front. Perl do not have have such well known libraries as far as I know.

> Python also got heavyweight libraries such as Numpy

This. For the kind of tasks data science involves, you either write C++ or you need the equivalent of Numpy. No Numpy means no Pandas, which means no sane way to wrangle datasets.

For all the hate it gets, Numpy is a world-class library.

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#16
post #8

Perl is expressive but the code can be hard to read. In general, Python is readable because it enforces the indentation and other language design choices. Python also got heavyweight libraries such as Numpy and Pandas which put it in the front. Perl do not have have such well known libraries as far as I know.

Perl seems to have died down just about everywhere. I've not used it professionally or seen it in use in well over a decade. It's so expressive and has so many ways to do things that reading other people's Perl can be challenging. I think this contributed to its downfall. And "other people" can often mean "yourself in six months" too.

> And "other people" can often mean "yourself in six months" too.

Amen to this point. I am currently the single engineer in a small software biz and after the last engineer left I thought to myself "well, at least I won't have to decipher his code anymore"...needless to say, I've been shocked by how much time I can spend on a system only to revisit it within a year and have little to no memory of how or why I did something a certain way!

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#17

tooling and network effects. perl doesn’t have the numerical chops to keep up, and if it started to fix that now, it has 20 years of headwind to fight through for probably marginal gains. Good luck.

Perl is also a nasty language to work with. Incredibly ugly. Even if perl was the standard for data science, I would be looking to escape it at every opportunity. In fact, that's what I did when I worked in a sector that had tooling and network effects for perl. I tried to escape using perl every chance I could.

There's R which is weird in a way like Perl is weird.

I think R is worse but it is bundled with a very good stats library. R users aren't so interested in "programming in the large" or even the small so it is OK.

Other than the meaningful indentation Python strikes me as a generic and normal programming language. I'm quite amused at how Python and Java got pattern matching at about the same time with a broadly similar approach to "fit this new feature in this language so that it feels like it belongs and plays well with decades of existing practice".

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#19
I love perl for regex scripts, where I need to quickly filter or transform a text file. I never liked it for other kinds of programming projects, for some reason to me it doesn’t feel as well suited to, say, numeric simulations.

> Naturally far more efficient

What does this mean exactly? One big reason for Python’s success in data science is numpy, which is far more efficient (especially on large data) than vanilla Python. I’m unaware of the state of the Perl ecosystem, does it have something similar?

> how come Perl lags far behind or gets dwarfed in data science by Python?

Perl seems to be lagging behind in general, no? Is there anything specific about data science where Perl should be shining?

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#20
Perl doesn't have the tooling/library ecosystem Python does for Data Science. Additionally lot of Data Science people come from maths/stats and Python is easier to begin with. In my experience most Data Scientists aren't full blown devs since you focus on different things (business aspects of what you do vs scalability).
Post reply on HN