Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python is not a great language for data science

blog.genesmindsmachines.com

Post reply on HN
Print view View original

201–210 of 339 posts

Re: Python is not a great language for data science

#201

Post by sheepscreek »

I really didn’t understand the author’s grievances. The only concrete example they illustrated was one where they concluded that Python without Pandas is verbose and ugly to achieve the same outcome, hence Python is not great for Data Science.

That’s a bad argument or a naive and obvious one; depending on how you look at it.

Python wasn’t designed for Data Science. It is not a DSL for it. MATLAB was arguably designed for scientific computing, and yet it’s the most disliked language in the StackOverflow liked/disliked index.

Here’s a different way to look at it. A good programming language is like the weather in a city. I would love to live somewhere where it’s 72F/23C all year round. But if it’s in the middle of nowhere and I’ve got no friends to hang out with, would I? I don’t think so.

FWIW, Python is like Sweden or Finland, with shitty weather for 6 months of the year yet thriving against all odds.

PS: I think the article’s topic is a bit click-batey (not a particularly useful discussion) because it’s polarizing and no one will be 100% right about it. It’s perhaps best thought of as an opinion piece.

Re: Python is not a great language for data science

#202

Post by Stratoscope »

Surac wrote: post #194

I at the moment try to learn python as a hobby language. I use c c++ and c# to earn my money. MY biggest problem is finding good examples that are up to date. I spent a whole day learning that there a four (I think) ways to do formatting strings. This „bloat“ in syntax makes even a simple print very heavy to digest. I don’t even bother using v2 python only v3. Also using whitespaces to block things together sounds ap…

You seem to be making things more difficult for yourself than they need to be.

For the strings, just use f-strings and forget all the others. You can even do things like this for debugging:

  >>> class User:
  ...     pass
  ... user = User()
  ... user.name = "Surac"
  ...
  >>> print(f"{user.name=}")
  user.name='Surac'
  >>>
For the block indenting, what editor are you using? Pretty much every modern editor lets you select a block and indent/unindent with Tab/Shift+Tab.

VS Code and PyCharm are both free and are great for Python coding. They each have a full debugger, which is invaluable when you are learning a language.

Re: Python is not a great language for data science

#203

Post by [deleted] »

whyenot wrote: post #29

What makes Python a great language for data science, is that so many people are familiar with it, and that it is an easy language to read. If you use a more obscure language like Clojure, Common Lisp, Julia, etc., many people will not be familiar with the language and unable to read or review your code. Peer review is fundamental to the scientific endeavor. If you only optimize on what is the best language for the ta…

[deleted]

Re: Python is not a great language for data science

#204

Post by flexagoon »

whyenot wrote: post #29

What makes Python a great language for data science, is that so many people are familiar with it, and that it is an easy language to read. If you use a more obscure language like Clojure, Common Lisp, Julia, etc., many people will not be familiar with the language and unable to read or review your code. Peer review is fundamental to the scientific endeavor. If you only optimize on what is the best language for the ta…

That's ok, I don't think anyone knows how to properly write Julia. After using it for a while and following the community (watching talks, checking the forum etc), I don't think it has a concept of code quality. You just throw random code at the wall until it starts working. Which makes sense, considering most of the users are scientists.

Re: Python is not a great language for data science

#205

Post by markkitti »

culebron21 wrote: post #176

This was underwhelming. I work with Python and Pandas, and I can show examples of much clumsier workflows I run into. The most often, you get dataframe[(dataframe.column1 == something) & ~dataframe.column2.isna()] constucts, which show that python syntax falls short here, and isn't suitable for such manipulations. Unfortunately, there's no alternative, and I don't see R as much easier, there are plenty of ugly things…

> Unfortunately, there's no alternative, and I don't see R as much easier, there are plenty of ugly things as well there.

Have you tried Polars? It really discourages the inefficient creation of intermediate boolean arrays such as in the code that you are showing.

> There's Julia -- it has serious drawbacks, like slow cold start if you launch a Julia script from the shell, which makes it unsuitable for CLI workflows.

Julia has gotten significantly better over time with regard to startup, especially with regard to plotting. There is definitely a preference for REPL or notebook based development to spread the costs of compilation over many executions. Compilation is increasingly modular with package based precompilation as well as ahead-of-time compilation modes. I do appreciate that typical compilation is an implicit step making the workflow much more similar to a scripting language than a traditionally compiled language.

I also do appreciate that traditional ahead-of-time static compilation to binary executable is also available now for deployment.

After a day of development in R or Python, I usually start regretting that I am not using Julia because I know yesterday's code could be executing much faster if I did. The question really becomes do I want to pay with time today or over the lifetime of the project.

Re: Python is not a great language for data science

#206

Post by procaryote »

lmm wrote: post #157
LtWorf wrote: post #100

Earlier quoted context omitted.

The success of python is due to not needing a broader ecosystem for A LOT of things. They are of course now abandoning this idea.

> The success of python is due to not needing a broader ecosystem for A LOT of things. I honestly think that was a coincidence. Perl and Ruby had other disadvantages, Python won despite having bad package management and a bloated standard library, not because of it.

The bloated standard library is the only reason I kept using python in spite of the packaging nightmare. I can do most things with no dependencies, or with one dependency I need over and over like matplotlib

If python had been lean and needed packages to do anything useful, while still having a packaging nightmare, it would have been unusable

Re: Python is not a great language for data science

#207

Post by IshKebab »

Surac wrote: post #194

I at the moment try to learn python as a hobby language. I use c c++ and c# to earn my money. MY biggest problem is finding good examples that are up to date. I spent a whole day learning that there a four (I think) ways to do formatting strings. This „bloat“ in syntax makes even a simple print very heavy to digest. I don’t even bother using v2 python only v3. Also using whitespaces to block things together sounds ap…

> but in reality you need to use editors that can indent and unindent whole blocks or I never get it right

What editor are you using that can't do that? Notepad?

Re: Python is not a great language for data science

#209

Post by neuropacabra »

I expected the author will complain rightfully about the tooling, including linters, formatters and package managers. Things improved drastically over the years with Astral’s ruff, uv and alpha stage ty.

But the article says that very exotic syntax is more readable. I think this is mostly about the libraries, where honestly I equally don’t like matplotlib and R’s ggplot. But I would not think it’s language problem.

I was hoping to find some performance benchmarks or something more than feelings about certain block of code. Don’t get me wrong I am also not a die hard fan of Python although I have written a lot or production code in it. Mentioning bloated, boilerplate code…I am afraid author should look on Java or any modern JavaScript project.

Re: Python is not a great language for data science

#210

Post by isolli »

aorist wrote: post #72

> Examples include converting boxplots into violins or vice versa, turning a line plot into a heatmap, plotting a density estimate instead of a histogram, performing a computation on ranked data values instead of raw data values, and so on. Most of this is not about Python, it’s about matplotlib. If you want the admittedly very thoughtful design of ggplot in Python, use plotnine > I would consider the R code to be sl…

Or seaborn. It was built exactly for this purpose: abstracting some of the annoying kinks of matplotlib while still offering a rich set of features.

https://seaborn.pydata.org/tutorial/introduction.html

Post reply on HN