Live data from Hacker News

Ask HN:Why is Perl so dwarfed in data science by Python?

news.ycombinator.com

91–100 of 120 posts

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#91

Data scientist here. Readable code is important in data science. People rely on our products to be based on solid numbers and logic, sometimes without any form of external validation. We can't just scribble line noise in a REPL until we get some output that looks vaguely reasonable. We need to be able to actually read the code and know that what it's doing makes sense. So readability matters. And Python is one of the…

Great last sentence, I’m going to steal it!

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#93
post #32

I do love Perl. However, for data science, the biggest drawback vs Python is how you work with complex data structures. Because everything in Python is an object, (de)referencing things is generally straightforward, like: somelist[12]['whatever']="abc" mylen=len(somelist) Where Perl devolves into a mess of sigils, when to use $ vs % vs @, wrapping references with {}, etc. $somelist[12]{'whatever'}="abc"; $mylen=lengt…

I used Perl for a lot data processing, mostly text. That time, our machine learning code was mostly C/C++. Perl slowly died before the big frameworks took off. It didn't compete with Python to be honest. Of course, even if there was a direct battle between Perl and Python, Python would probably win anyway.

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#94
post #32

I do love Perl. However, for data science, the biggest drawback vs Python is how you work with complex data structures. Because everything in Python is an object, (de)referencing things is generally straightforward, like: somelist[12]['whatever']="abc" mylen=len(somelist) Where Perl devolves into a mess of sigils, when to use $ vs % vs @, wrapping references with {}, etc. $somelist[12]{'whatever'}="abc"; $mylen=lengt…

I used Perl for a lot data processing, mostly text. That time, our machine learning code was mostly C/C++. Perl slowly died before the big frameworks took off. It didn't compete with Python to be honest. Of course, even if there was a direct battle between Perl and Python, Python would probably win anyway.

I do find Perl much easier for manipulating text files, file names, and the sorts of things where bash or awk are too tedious. Then there's a line where something like Python becomes more attractive. Perl is still a lot faster than Python for many tasks too...the Perl devs seems to spend more time optimizing certain paths. Or perhaps "everything is an object" in Python slows everything down?

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#95
Data science involves building many statistical models, visualizing complex data structures in many ways, and summarizing results into "pretty" figures.

In this way, the statistical packages and plotting software in Python is better than Perl. I would say R is even better than Python and Perl for certain statistical analyses and quickly plotting complex data in different ways.

Perl might beat other languages in wrangling certain types of data like comma separated values or other 2D arrays in terms of writing expressive one-liners. Perhaps that is what you mean by "naturally efficient"? How much one can do in one line of code?

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#96
post #67

As someone who wrote a lot of Perl in the late 90's and early 2000's and moved to Python for scientific work (simulations, etc.) as well as other tasks (web apps, etc), what I saw was: - Perl actually was really popular for a while back then, especially in the Bioinformatics/Genomics space. It was all over that field, I think partially because it's really easy to think of a genome as just a text string of ATCGs and P…

I built and maintained the python stack at a financial firm that had a sizeable investment in perl and PDL, and iirc PDL was unreliable (crashy, not sure about other aspects of reliability) and I don't recall that it was making the transition to 64 bit in the mid/late 2000s at the speed numpy was. Do you remember any more detail from the perl side of the fence?

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#99
Python spends most of it's time in C extensions for most applications, so performance isn't as big of an issue.

Perl is.... perl. If you read it it's not at all obvious what's happening. They don't believe in the one and only one obvious way principle.

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#100
post #67

As someone who wrote a lot of Perl in the late 90's and early 2000's and moved to Python for scientific work (simulations, etc.) as well as other tasks (web apps, etc), what I saw was: - Perl actually was really popular for a while back then, especially in the Bioinformatics/Genomics space. It was all over that field, I think partially because it's really easy to think of a genome as just a text string of ATCGs and P…

> especially in the Bioinformatics/Genomics space

My first internship was re-writing a Perl program that parsed the output of `samtools mpileup`[0] into a C program that used htslib[1] to directly read the BAM file and extract the relevant data. This preprocessing (the Perl version) was the slowest part of the pipeline, even slower than the actual data analysis.

[0]: http://www.htslib.org/doc/samtools-mpileup.html [1]: http://www.htslib.org/

Post reply on HN