Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python is not a great language for data science

blog.genesmindsmachines.com

Post reply on HN
Print view View original

91–100 of 339 posts

Re: Python is not a great language for data science

#91

Post by ashdev »

progval wrote: post #85

The pure Python code in the last example is more verbose than it needs to be. groups = {} for row in filtered: key = (row['species'], row['island']) if key not in groups: groups[key] = [] groups[key].append(row['body_mass_g']) can be rewritten as: groups = collections.defaultdict(list) for row in filtered: groups[(row['species'], row['island'])].append(row['body_mass_g']) and variance = sum((x - mean) ** 2 for x in v…

Disagree.

In the first instance, the original code is readable and tells me exactly what's what. In your example, you're sacrificing readability for being clever.

Clear code(even if verbose) is better than being clever.

Re: Python is not a great language for data science

#92

Post by hekkle »

For those who thought the article was TL;DR, the author argues.

- A General programming language like Python is good enough for data science but isn't specifically designed for it.

- A language that is specifically designed for Data Science like R is better at Data Science.

Who would have thought?

Re: Python is not a great language for data science

#94

Post by psunavy03 »

I'm not sure what that last example is meant to be other than an anti-Python caricature. If you're implementing calculating things like standard deviations by hand, that's not real-world coding, that's the undergraduate harassment package which should end with a STEM bachelor's.

Of course there's a bunch of loops and things; you're exposing what has to happen in both R and Python under the hood of all those packages.

Re: Python is not a great language for data science

#95

Post by riskassessment »

RobinL wrote: post #16

I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…

> R is perhaps the closest, because it has data.frame as a 'first class citizen', but most people don't seem to use it, and use e.g. tibbles from dplyr instead.

Everyone in R uses data.frame because tibble (and data.table) inherits from data.frame. This means that "first class" (base R) functions work directly on tibble/data.table. It also makes it trivial to convert between tibble, data.table, and data.frames.

Re: Python is not a great language for data science

#96

Post by 1vuio0pswjnm7 »

npalli wrote: post #80
aorist wrote: post #72

> Examples include converting boxplots into violins or vice versa, turning a line plot into a heatmap, plotting a density estimate instead of a histogram, performing a computation on ranked data values instead of raw data values, and so on. Most of this is not about Python, it’s about matplotlib. If you want the admittedly very thoughtful design of ggplot in Python, use plotnine > I would consider the R code to be sl…

Python is nothing without it’s batteries.

What language is used to write the batteries

Re: Python is not a great language for data science

#97

Post by ModernMech »

RobinL wrote: post #16

I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…

It makes sense from a historical perspective. Tables are a thing in many languages, just not the ones that mainstream devs use. In fact, if you rank programming languages by usage outside of devs, the top languages all have a table-ish metaphor (SQL, Excel, R, Matlab).

The languages devs use are largely Algol derived. Algol is a language that was used to express algorithms, which were largely abstractions over Turing machines, which are based around an infinite 1D tape of memory. This model of 1D memory was built into early computers, and early operating systems and early languages. We call it "mechanical sympathy".

Meanwhile, other languages at the same time were invented that weren't tied so closely to the machine, but were more for the purpose of doing science and math. They didn't care as much about this 1D view of the world. Early languages like Fortran and Matlab had notions of 2D data matrices because math and science had notions of 2D data matrices. Languages like C were happy to support these things by using an array of pointers because that mapped nicely to their data model.

The same thing can be said for 1-based and 0-based indexing -- languages like Matlab, R, and Excel are 1-based because that's how people index tables; whereas languages like C and Java are 0-based because that's how people index memory.

Re: Python is not a great language for data science

#98

Post by billyoyo »

ashdev wrote: post #91
progval wrote: post #85

The pure Python code in the last example is more verbose than it needs to be. groups = {} for row in filtered: key = (row['species'], row['island']) if key not in groups: groups[key] = [] groups[key].append(row['body_mass_g']) can be rewritten as: groups = collections.defaultdict(list) for row in filtered: groups[(row['species'], row['island'])].append(row['body_mass_g']) and variance = sum((x - mean) ** 2 for x in v…

Disagree. In the first instance, the original code is readable and tells me exactly what's what. In your example, you're sacrificing readability for being clever. Clear code(even if verbose) is better than being clever.

Using a very common utility in the standard library is to avoid reinventing the wheel is not "clean code"?

defaultdict is ubiquitous in modern python, and is far from a complicated concept to grasp.

Re: Python is not a great language for data science

#99

Post by hekkle »

whyenot wrote: post #29

What makes Python a great language for data science, is that so many people are familiar with it, and that it is an easy language to read. If you use a more obscure language like Clojure, Common Lisp, Julia, etc., many people will not be familiar with the language and unable to read or review your code. Peer review is fundamental to the scientific endeavor. If you only optimize on what is the best language for the ta…

> What makes Python a great language for data science, is that so many people are familiar with it

While I agree with you in principal this also leads to what I call the "VB Effect". Back in the day VB was taught at every school as part of the standard curriculum. This made every kid a 'computer wizz'. I have had to fix many a legacy codebase that was started by someone's nephew the whizz kid.

Re: Python is not a great language for data science

#100

Post by LtWorf »

pphysch wrote: post #89
npalli wrote: post #80

Earlier quoted context omitted.

Python is nothing without it’s batteries.

The design and success of e.g. Golang is pretty strong support for the idea that you can't and shouldn't separate a language from its broader ecosystem of tooling and packages.

The success of python is due to not needing a broader ecosystem for A LOT of things.

They are of course now abandoning this idea.

Post reply on HN