Perl was already going the way of the dodo in late 2000s. Perl 6 finished it off.
Ask HN:Why is Perl so dwarfed in data science by Python?
101–110 of 120 posts
Re: Ask HN:Why is Perl so dwarfed in data science by Python?
#102Data scientist here. Readable code is important in data science. People rely on our products to be based on solid numbers and logic, sometimes without any form of external validation. We can't just scribble line noise in a REPL until we get some output that looks vaguely reasonable. We need to be able to actually read the code and know that what it's doing makes sense. So readability matters. And Python is one of the…
Readability is really a weak argument. I've seen a lot of unreadable code written by data scientist in Python. And a lot of readable Perl code. It's not about the language, it's about the best practices. The concept behind sigils is very simple and once you've learned it, it's not that threatening. And when talking about data structures, it is a benefit to have it more structured and know that you are dealing with an…
> Readability is really a weak argument.
I strongly disagree.
As someone who had to interview for VLSI design positions (both sides of the table) back when Perl was popular, I used to take a printout of a small tool I developed in Perl with me to every interview (parse a VCD file and check the results).
This forced the interviewers into MY subset of Perl. That made a huge difference into how competent they thought I was with Perl.
Most of the interviewers gave up after we went through about 10-20 lines of code because they had never seen half of the stuff in my code--and I wasn't doing anything particularly obscure (decorate-sort-undecorate/Schwartzian transform would lose most of them).
Re: Ask HN:Why is Perl so dwarfed in data science by Python?
#103C is more efficient than both of them. For that matter, so are tons of other languages. For that matter, I'm not sure what you mean when you say Perl is "naturally more efficient" than Python. There's nothing about Perl that makes it easy to run faster, and the ability to write incomprehensible one-liners is not a very satisfying measure of "efficiency". As for why people choose Python over Perl, Perl is a pain in th…
The design decisions of the Python language (arbitrary length integers for example) lead Python to just be SLOW. It's much slower than Perl.
Don't believe me? Look at how numbers are stored: https://rushter.com/blog/python-integer-implementation/
But it doesn't really matter. Computers increase in speed over time. Libraries allow speed sensitive sections to use more sane data types. As you pointed out, the Perl language's design decisions cause much worse problems -- hard to identify bugs.
Re: Ask HN:Why is Perl so dwarfed in data science by Python?
#104As someone who wrote a lot of Perl in the late 90's and early 2000's and moved to Python for scientific work (simulations, etc.) as well as other tasks (web apps, etc), what I saw was: - Perl actually was really popular for a while back then, especially in the Bioinformatics/Genomics space. It was all over that field, I think partially because it's really easy to think of a genome as just a text string of ATCGs and P…
> especially in the Bioinformatics/Genomics space My first internship was re-writing a Perl program that parsed the output of `samtools mpileup`[0] into a C program that used htslib[1] to directly read the BAM file and extract the relevant data. This preprocessing (the Perl version) was the slowest part of the pipeline, even slower than the actual data analysis. [0]: http://www.htslib.org/doc/samtools-mpileup.html [1…
oof I can't imagine converting perl string manipulation to C was fun. Did you try to replicate what the perl code was doing, or did you just redo the entire thing in a more C like way?
Re: Ask HN:Why is Perl so dwarfed in data science by Python?
#105Earlier quoted context omitted.
We'll have to agree to disagree on most of this, which is fine, better than having a stupidity-driven argument. I'm sure the aspects of Perl that are weird and gross to me are great for some other people. > Perl was extensively used by scientific community, but Python was the lucky one that was backed by Google in 2005. I'm curious about this. Can you point to any specific evidence that being "backed by Google in 200…
> I'm curious about this. Can you point to any specific evidence that being "backed by Google in 2005" was a momentous event for Python? Here are some stats with the popularity of various languages over time: https://statisticsanddata.org/data/the-most-popular-programm... In 2005 Google hired Guido van Rossum, where he spent half of his time developing the Python language. As you can see, before 2006, Python was litt…
The sibling comment mentioned that there's probably no contribution from Guido regarding the meteoric rise of Python in data science but the truth it's really the opposite [1]. Guido was really instrumental in suitability of Python for data science [2].
To quote from the magazine article on Python for Scientists and Engineers:
"During these early years, there was considerable interaction between the standard and scientific Python communities. In fact, Guido van Rossum, Python's Benevolent Dictator For Life (BDFL), was an active member of the matrix-sig. This close interaction resulted in Python gaining new features and syntax specifically needed by the scientific Python community. While there were miscellaneous changes, such as the addition of complex numbers, many changes focused on providing a more succinct and easier to read syntax for array manipulation. For instance, the parenthesis around tuples were made optional so that array elements could be accessed through, for example, a[0,1] instead of a[(0,1)]. The slice syntax gained a step argument— a[::2] instead of just a[:], for example—and an ellipsis operator, which is useful when dealing with multidimensional data structures."
Personally I'd much prefer D language to replace python for data science due to its Python like programming vibes but with more powerful real-time programming constructs, and its suitability for beginners or type A data scientists [3]. Python is already showing its limitations especially for the B type of data scientists [4]. There's recent discussions on Dlang forum for the potential improvements of D for data science and there's also recommendation for Walter to join such SIG as was Guido [5].
[1]Numpy History:
https://en.wikipedia.org/wiki/NumPy#History
There's even recomendation from Dlang forum for Walter to join such SIG [3].
[2]Python for Scientists and Engineers:
https://www.computer.org/csdl/magazine/cs/2011/02/mcs2011020...
[3]Teaching D from Scratch: Is it a viable first language?
https://news.ycombinator.com/item?id=29711800
[4]Type A Data Scientist vs. Type B Data Scientist:
https://www.projectpro.io/article/type-a-data-scientist-vs-t...
[5]Dlang forum: Pandas like features:
https://forum.dlang.org/thread/zuebxhkycgxsoipsgnbk@forum.dl...
Re: Ask HN:Why is Perl so dwarfed in data science by Python?
#106Earlier quoted context omitted.
> I'm curious about this. Can you point to any specific evidence that being "backed by Google in 2005" was a momentous event for Python? Here are some stats with the popularity of various languages over time: https://statisticsanddata.org/data/the-most-popular-programm... In 2005 Google hired Guido van Rossum, where he spent half of his time developing the Python language. As you can see, before 2006, Python was litt…
The sibling comment mentioned that there's probably no contribution from Guido regarding the meteoric rise of Python in data science but the truth it's really the opposite [1]. Guido was really instrumental in suitability of Python for data science [2]. To quote from the magazine article on Python for Scientists and Engineers: "During these early years, there was considerable interaction between the standard and scie…
I'm sure Guido did things for the scientific community. My claim was that he probably didn't move the needle for them after coming to Google in 2005.
The other person claimed that Google magic somehow led to Python becoming dominant in the sci/numeric/data computing community, and that just strikes me as wildly implausible. Google has done negligible foundational work on scientific Python. No one in that community has ever cared what Google thinks.
Anyways, I'm curious why you think D has potential as a Data Science language? Why not a more popular language like Julia or Rust?
Re: Ask HN:Why is Perl so dwarfed in data science by Python?
#107Because it was used by a newsroom in Lawrence, Kansas and they wrote a web framework in python called Django. Their journalistic culture led Django to have really great documentation, which led to an influx of more casual developers who had an expectation of great documentation to make up for the fact that their main expertise was elsewhere. This made libraries with good docs more successful, making it easier for university courses to choose python as an initial language. This made it easier for university labs to agree on python as a language. This led Travis Oliphant & friends to develop numpy etc.
Re: Ask HN:Why is Perl so dwarfed in data science by Python?
#108Earlier quoted context omitted.
Readability is really a weak argument. I've seen a lot of unreadable code written by data scientist in Python. And a lot of readable Perl code. It's not about the language, it's about the best practices. The concept behind sigils is very simple and once you've learned it, it's not that threatening. And when talking about data structures, it is a benefit to have it more structured and know that you are dealing with an…
> Readability is really a weak argument. I strongly disagree. As someone who had to interview for VLSI design positions (both sides of the table) back when Perl was popular, I used to take a printout of a small tool I developed in Perl with me to every interview (parse a VCD file and check the results). This forced the interviewers into MY subset of Perl. That made a huge difference into how competent they thought I…
I guess I'm showing my youth here but I never imagined that sorting on a computed key was once known by such an exalted title as Schwartzian Transform. This is basic, built-in functionality in Python.
Re: Ask HN:Why is Perl so dwarfed in data science by Python?
#109Perl is what I am most proficient in, and have already completed various AI projects with, but my colleagues tell me it will be worth it to learn how to program in python, even though I will be set back in the short term.
Re: Ask HN:Why is Perl so dwarfed in data science by Python?
#110Earlier quoted context omitted.
> Readability is really a weak argument. I strongly disagree. As someone who had to interview for VLSI design positions (both sides of the table) back when Perl was popular, I used to take a printout of a small tool I developed in Perl with me to every interview (parse a VCD file and check the results). This forced the interviewers into MY subset of Perl. That made a huge difference into how competent they thought I…
I guess I'm showing my youth here but I never imagined that sorting on a computed key was once known by such an exalted title as Schwartzian Transform. This is basic, built-in functionality in Python.
It's not just that the key is computed, but that the key has an internal sort comparison which is way faster than a user code comparison. It was an example of the fact that "Big-Oh notation is nice, but constants sometimes matter."
Both sorts have the same O(n log n) number of operations performance. However, in Perl, using an internal comparison for integers was a significant constant faster than using a user specified comparison function--at least an order of magnitude and sometimes two. Consequently, the 2*O(n) to decorate and undecorate the list with integers to sort by got swamped by the 1/10*O(n log n) sort improvement.
I believe that this overhead eventually got reduced. However, I remember using this a LOT in Perl 4 and early Perl 5 days.