Live data from Hacker News

Ask HN:Why is Perl so dwarfed in data science by Python?

news.ycombinator.com

111–120 of 120 posts

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#111
post #32

I do love Perl. However, for data science, the biggest drawback vs Python is how you work with complex data structures. Because everything in Python is an object, (de)referencing things is generally straightforward, like: somelist[12]['whatever']="abc" mylen=len(somelist) Where Perl devolves into a mess of sigils, when to use $ vs % vs @, wrapping references with {}, etc. $somelist[12]{'whatever'}="abc"; $mylen=lengt…

Some people don't like sigils on variable names, and I get that. Personally I find them useful.

As far as the perldsc page example, I think that comes down to having a better personal style that makes your code readable. I would make the top level variable a hash reference rather than a plain hash, and purposefully not insist on dereferencing multiple levels all on one line.

As someone pointed out in another comment, you can make the Python version look just as lousy by virtue of a lousy style.

My version of code doing the same task would be something like this:

  my $familyData = $TV->{$family};
  my $lead       = $familyData->{lead};
  my $kidsList   = $familyData->{kids};

  my $kidCount = scalar( @{ $kidsList } );

  my $kidListStr = join (
    ", ",
    map { $_->{name} } @{ $kidsList }
  );

  print "it turns out that $lead has ";
  print "$kidCount kids named $kidListStr\n";

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#112

Earlier quoted context omitted.

You can write hard-to-read code in any language. You can't write easy-to-read code in any language. Python tends to make easy-to-read code easier to write than Perl does.

> You can't write easy-to-read code in any language. I've seen (rarely) old shell scripts and even assembler made readable by copious use of comments. I think the claim should rather be that you can't write easy-to-read self-documenting code in all languages.

The issue is with data movement, in my opinion. Heavily pipelined command languages like shell and stack languages tend towards unreadable because they perform actions on arbitrary data with implicit transfers between functions, which can be confusing to the uninitiated, because you have to know every (not necessarily obvious) command and it's parameters beforehand.

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#113
post #67

As someone who wrote a lot of Perl in the late 90's and early 2000's and moved to Python for scientific work (simulations, etc.) as well as other tasks (web apps, etc), what I saw was: - Perl actually was really popular for a while back then, especially in the Bioinformatics/Genomics space. It was all over that field, I think partially because it's really easy to think of a genome as just a text string of ATCGs and P…

I built and maintained the python stack at a financial firm that had a sizeable investment in perl and PDL, and iirc PDL was unreliable (crashy, not sure about other aspects of reliability) and I don't recall that it was making the transition to 64 bit in the mid/late 2000s at the speed numpy was. Do you remember any more detail from the perl side of the fence?

Not really. I vaguely remember trying out PDL and not getting very far. I was pretty comfortable writing C back then, so any time I'd run into performance issues in Perl, I'd usually just switch to C. Getting PDL working properly was painful enough that it didn't really seem worth spending a lot of time on when I could just write a small C program, especially for commandline data pipeline type situations. Once I started hitting projects with requirements for interactive GUIs though, keeping everything in the same language becomes more valuable and Python had the better complete package.

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#114
post #100
post #67

As someone who wrote a lot of Perl in the late 90's and early 2000's and moved to Python for scientific work (simulations, etc.) as well as other tasks (web apps, etc), what I saw was: - Perl actually was really popular for a while back then, especially in the Bioinformatics/Genomics space. It was all over that field, I think partially because it's really easy to think of a genome as just a text string of ATCGs and P…

> especially in the Bioinformatics/Genomics space My first internship was re-writing a Perl program that parsed the output of `samtools mpileup`[0] into a C program that used htslib[1] to directly read the BAM file and extract the relevant data. This preprocessing (the Perl version) was the slowest part of the pipeline, even slower than the actual data analysis. [0]: http://www.htslib.org/doc/samtools-mpileup.html [1…

Yeah, I feel like that was a pretty common workflow. Someone would get a new algorithm working in Perl, test it on fairly small datasets, maybe publish their paper, then some poor chump would have to carefully rewrite it in C to make it actually useful for real data. I was at a university, so our chumps were grad students. I'm sure interns were the industry equivalent :)

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#115
The Perl community is horrible, that is the real reason. Criticism about Perl readability and other Perl language meme critics are easily debunked myths. the Perl language semantics and logic is fantastic and easily readable after you LEARN it. The Perl interpreter is fantastic, performs great and have very helpful warnings and strictures.

However the community is untalented, they historically produce very bad code, it all started with Matt's script archive. They do create horrible sites, PerlMonks is a good example, It looks horrible and its usability is horrible. The code posted by PerlMonks users is mostly very bad. CPAN that is often pointed as positive is just a mostly poorly documented repository of very bad code. There are very few usable modules on CPAN.

Perl community has also written some of the worst technical books ever.

Talent attracts talent, that is the reason Perl is dead for any kind of usage and Python is popular.

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#116
post #110

Earlier quoted context omitted.

I guess I'm showing my youth here but I never imagined that sorting on a computed key was once known by such an exalted title as Schwartzian Transform. This is basic, built-in functionality in Python.

It's not just that the key is computed, but that the key has an internal sort comparison which is way faster than a user code comparison. It was an example of the fact that "Big-Oh notation is nice, but constants sometimes matter." Both sorts have the same O(n log n) number of operations performance. However, in Perl, using an internal comparison for integers was a significant constant faster than using a user specif…

Sounds generally plausible to me. I learned all sorts of obscure tricks to make MATLAB run faster in my grad school days... I wouldn't be surprised if something like this was useful over there as well.

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#117
post #111
post #32

I do love Perl. However, for data science, the biggest drawback vs Python is how you work with complex data structures. Because everything in Python is an object, (de)referencing things is generally straightforward, like: somelist[12]['whatever']="abc" mylen=len(somelist) Where Perl devolves into a mess of sigils, when to use $ vs % vs @, wrapping references with {}, etc. $somelist[12]{'whatever'}="abc"; $mylen=lengt…

Some people don't like sigils on variable names, and I get that. Personally I find them useful. As far as the perldsc page example, I think that comes down to having a better personal style that makes your code readable. I would make the top level variable a hash reference rather than a plain hash, and purposefully not insist on dereferencing multiple levels all on one line. As someone pointed out in another comment,…

[deleted]

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#118
post #60

Data scientist here. Readable code is important in data science. People rely on our products to be based on solid numbers and logic, sometimes without any form of external validation. We can't just scribble line noise in a REPL until we get some output that looks vaguely reasonable. We need to be able to actually read the code and know that what it's doing makes sense. So readability matters. And Python is one of the…

Readability is really a weak argument. I've seen a lot of unreadable code written by data scientist in Python. And a lot of readable Perl code. It's not about the language, it's about the best practices. The concept behind sigils is very simple and once you've learned it, it's not that threatening. And when talking about data structures, it is a benefit to have it more structured and know that you are dealing with an…

Unless you are hell-bent on writing obfuscated code, Python code is almost always readable. Even when I didn't know any Python or any other language, I could make an educated guess what some Python program is doing. However, I have seen some of Perl code and I was like wtf.

As for best practices, Python does insist on writing readable code a lot. Python people always nudge you to write readable code. Guido himself said he designed Python to be a readable language.In general, Python has the philosophy of one way of doing something which contrasts with Perl way where you can accomplish a task in numerous ways. You can't be too clever with Python (little exaggerated position). However, the same is not true with Perl.

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#119
Data science is primarily about communication, both of your analysis to other people, including laypeople, domain experts, other data scientists, ops folks ... and those same people communicating with you about their needs, important context, expectations of SLAs or constraints ...

The main barriers to good communication of data science is the translation between the real world goals, data, algorithms, assumptions and priors, scientific and statistical methods being applied, testing, the deployed model's performance and ongoijg monitoring and management, etc .. and the actual code and artifacts produced to achieve those ends.

Python wins because there is a wide ecosystem around those translation efforts.

* Visualization and statistical profiling are first class citizens, probably the killer app in terms of communicating difficult mathematical concepts

* Easy to extend, very "framework-friendly", so you can "speak the same language" between data engineering, DS, analysts, MLOps folks

* Needless to say, network effects of a community used to Pythonic idioms

Re: Ask HN:Why is Perl so dwarfed in data science by Python?

#120
post #36

Earlier quoted context omitted.

That is the historical fact, yes. However have you looked at Raku [0] / Perl6? It is mind boggling. 0, https://www.raku.org/

is it mind boggling? depends on what you mean Every time I look at an example I am turned off and my mind is in a way "boggled" ... I just opened the tutorial on the Raku site I see this say looks_like_number "foo"; I find this awful, right from the beginning, the say turns me off, computers do not speak, at least I really hope the computer is not actually speaking when I type that. Then there are the weird rules of…

  > right from the beginning, the say turns me off, computers do not speak
This seems a bit of an odd argument. You've never heard the phrase "The sign says ...", or asked someone "What does the error message say?"

That said, `say` is just a normal function, and you could always bind a new function and use that instead.

Post reply on HN