Live data from Hacker News

Python is not a great language for data science

blog.genesmindsmachines.com

81–90 of 339 posts

Re: Python is not a great language for data science

#82
post #80
post #72

> Examples include converting boxplots into violins or vice versa, turning a line plot into a heatmap, plotting a density estimate instead of a histogram, performing a computation on ranked data values instead of raw data values, and so on. Most of this is not about Python, it’s about matplotlib. If you want the admittedly very thoughtful design of ggplot in Python, use plotnine > I would consider the R code to be sl…

Python is nothing without it’s batteries.

Python is its batteries.

Re: Python is not a great language for data science

#83
post #16

I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…

[deleted]

Re: Python is not a great language for data science

#84
I'm heavily inclined to agree with the general thought, but I balk at the low level code showing why a language is bad at something. In this specific case, without the tidyverse, R isn't exactly peaches and cream.

As annoying as it is to admit it, python is a great language for data science almost strictly because it has so many people doing data science with it. The popularity is, itself, a benefit.

Re: Python is not a great language for data science

#85
The pure Python code in the last example is more verbose than it needs to be.

    groups = {}
    for row in filtered:
        key = (row['species'], row['island'])
        if key not in groups:
            groups[key] = []
        groups[key].append(row['body_mass_g'])
can be rewritten as:

    groups = collections.defaultdict(list)
    for row in filtered:
        groups[(row['species'], row['island'])].append(row['body_mass_g'])
and

    variance = sum((x - mean) ** 2 for x in values) / (n - 1)
    std_dev = math.sqrt(variance)
as:

    std_dev = statistics.stddev(values)

Re: Python is not a great language for data science

#86
> I think people way over-index Python as the language for data science. It has limitations that I think are quite noteworthy. There are many data-science tasks I’d much rather do in R than in Python.

R is kind of a super-specialized language. Python is much more general purpose.

R failed to evolve, let's be honest. Python won via jupyter - I see this used ALL the time in universities. R is used too, but mostly for statistics related courses only, give or take.

Perhaps R is better for its niche, but Python has more momentum and in thus, dominates over R. That's simply the reality of the situation. It is like the bulldozer moving forward, at a fast speed.

> I say “This is great, but could you quickly plot the data in this other way?”

Ok so ... he would have to adjust R code too, right? And finding good info on that is simply harder. He says he has experience with universities. Well, I do too, and my experience is that people are WAY better with python than with R. You simply see that more students will drop out from R than from python. That's also simply the reality of the situation.

> They appear to be sufficiently cumbersome or confusing that requests that I think should be trivial frequently are not.

I am sure the reverse also applies. Pick some python library, do something awesome, then tell the R students to do the same. I bet he will have the same problems.

> So many times, I felt that things that would be just a few lines of simple R code turned out to be quite a bit longer and fairly convoluted.

Ok, so here he is trolling. Flat out - I said it.

I wrote a LOT of python and quite a bit of R. There is no way in life that the R code is more succinct than the python code for about 90% of the use cases out there. Sorry, that's simply not the case. R is more verbose.

> Here is the relevant code in R, using the tidyverse approach:

    penguins |>
      filter(!is.na(body_mass_g)) |>
      group_by(species, island) |>
      summarize(
This is like perl. They also don't adapt. R is going to lose grounds.

This professor just hasn't realised that he is slowly becoming a fossil himself, by being unable to see that x is better than y.

Re: Python is not a great language for data science

#87
Refuses to learn tool so tool is broken... There is no problem with python for this. If you hate boiler plate job the club, get llms to generate it for you and move on to doing real work (or get involved in improving the language or libraries directly)

Re: Python is not a great language for data science

#88
post #16

I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…

Because there's no obvious universal optimal data structure for heterogeneous N-dimensional data with varying distributions? You can definitely do that, but it requires an order of magnitude more resource use as baseline.

Re: Python is not a great language for data science

#89
post #80
post #72

> Examples include converting boxplots into violins or vice versa, turning a line plot into a heatmap, plotting a density estimate instead of a histogram, performing a computation on ranked data values instead of raw data values, and so on. Most of this is not about Python, it’s about matplotlib. If you want the admittedly very thoughtful design of ggplot in Python, use plotnine > I would consider the R code to be sl…

Python is nothing without it’s batteries.

The design and success of e.g. Golang is pretty strong support for the idea that you can't and shouldn't separate a language from its broader ecosystem of tooling and packages.
Post reply on HN