The pure Python code in the last example is more verbose than it needs to be. groups = {} for row in filtered: key = (row['species'], row['island']) if key not in groups: groups[key] = [] groups[key].append(row['body_mass_g']) can be rewritten as: groups = collections.defaultdict(list) for row in filtered: groups[(row['species'], row['island'])].append(row['body_mass_g']) and variance = sum((x - mean) ** 2 for x in v…
Disagree. In the first instance, the original code is readable and tells me exactly what's what. In your example, you're sacrificing readability for being clever. Clear code(even if verbose) is better than being clever.
Python is not a great language for data science
131–140 of 339 posts
Re: Python is not a great language for data science
#132Re: Python is not a great language for data science
#133> Python is pretty good for deep learning. There’s a reason PyTorch is the industry standard. When I’m talking about data science here, I’m specifically excluding deep learning.
I've written very little deep learning code over my career, but made very frequent use of the GPU and differentiable programming for non-deep learning specific tasks. In general Python is much easier to write quantitative programs that make use of the hardware, and you have a lot more options when your problem doesn't fit into RAM.
> I have been running a research lab in computational biology for over two decades.
I've been working nearly exclusively in industry for these two decades and a major reason I find Python just better is it's much, much easier to interface with other parts of engineering when you're a using truly general purpose PL. I've actually never worked for a pure Python shop, but it's generally much easier to get production ML/DS solutions into prod when working with Python.
> Data science as I define it here involves a lot of interactive exploration of data and quick one-off analyses or experiments
This re-iterates the previous difference. In my experience I would call this "step one" in all my DS related work. The first step is to understand the problem and de-risk. But the vast majority of code and work is related to delivering a scalable product.
You can say that's not part of "data science", but if you did you'd have a hard time finding a job on most of the teams I've worked on.
All that said, my R vs Python experience has boiled down to: If your end result is a PDF report, R is superior. If your end result is shipping a product, then Python is superior. And my experience has been that, outside of university labs, there aren't a lot of jobs out there for DS folks who only want to deliver PDFs.
Re: Python is not a great language for data science
#134Earlier quoted context omitted.
I can't speak for Julia - never used it; never used Common Lisp for analyzing data (I don't think it's very "data-oriented" for the modern age and the shape of data), but Clojure is really not "obscure" - it only looks weird for the first fifteen minutes or so; once you start using it - it is one of the most straightforward and reasonable languages out there - it is in fact simpler than Python and Javascript. Immutab…
I tried to get into Clojure, but a lot of the JVM hosted languages require some Java experience. Same thing with Scala and Kotlin or F# on .NET. The early tooling was also pretty dependent on Vim or Emacs. Maybe it's all easier now with VSCode or something like that.
The tooling story is also very solid - I use Emacs, but many of my friends and colleagues use IntelliJ, Vim, Sublime and VSCode, and some of them migrated to it from Atom.
Re: Python is not a great language for data science
#135Earlier quoted context omitted.
Disagree. In the first instance, the original code is readable and tells me exactly what's what. In your example, you're sacrificing readability for being clever. Clear code(even if verbose) is better than being clever.
Using a very common utility in the standard library is to avoid reinventing the wheel is not "clean code"? defaultdict is ubiquitous in modern python, and is far from a complicated concept to grasp.
The difference between the examples is so trivial I'm not really sure why the parent comment felt compelled to complain.
Re: Python is not a great language for data science
#136Earlier quoted context omitted.
Python is nothing without it’s batteries.
I hear this so much from Python people -- almost like they are paid by the word to say it. Is it different from Perl, Ruby, Java, or C# (DotNet)? Not in my experience, except people from those communities don't repeat that phrase so much. The irony here: We are talking about data science. 98% of "data science" Python projects start by creating a virtual env and adding Pandas and NumPy which have numerous (really: squ…
pandas==2.3.3
├── numpy [required: >=1.22.4, installed: 2.2.6]
├── python-dateutil [required: >=2.8.2, installed: 2.9.0.post0]
│ └── six [required: >=1.5, installed: 1.17.0]
├── pytz [required: >=2020.1, installed: 2025.2]
└── tzdata [required: >=2022.7, installed: 2025.2]
Re: Python is not a great language for data science
#137I'm not sure what that last example is meant to be other than an anti-Python caricature. If you're implementing calculating things like standard deviations by hand, that's not real-world coding, that's the undergraduate harassment package which should end with a STEM bachelor's. Of course there's a bunch of loops and things; you're exposing what has to happen in both R and Python under the hood of all those packages.
It's pretty clear the post is focused on the context of work being done in an academic research lab. In that context I think most of the points are pretty valid, but most of the real world benefit I've experience from using Python is being able to work more closely with engineering (even on non-Python teams).
I shipped R code to a production environment once over my career and it felt incredibly fragile.
R is great for EDA, but really doesn't work well for iteratively building larger software projects. R is has a great package system, but it's not so great when you need abstraction in between.
Re: Python is not a great language for data science
#138Earlier quoted context omitted.
Python has a list of issues fundamentally broken in the language, and relies heavily on integrated library bindings to operate at reasonable speeds/accuracy. Julia allows embedding both R and Python code, and has some very nice tools for drilling down into datasets: https://www.queryverse.org/ It is the first language I've seen in decades that reduces entire paradigms into single character syntax, often outperforming…
Deeply ironic for a Julia proponent to smear a popular language as "fundamentally broken" without evidence. https://yuri.is/not-julia/
Not a "smear", but rather a well known limitation of the language. Perhaps your environment context works differently than mine.
It is bizarre people get emotionally invested in something so trivial and mundane. Julia is at v1.12.2 so YMMV, but Queryverse is a lot of fun =3
Re: Python is not a great language for data science
#139Earlier quoted context omitted.
I can't speak for Julia - never used it; never used Common Lisp for analyzing data (I don't think it's very "data-oriented" for the modern age and the shape of data), but Clojure is really not "obscure" - it only looks weird for the first fifteen minutes or so; once you start using it - it is one of the most straightforward and reasonable languages out there - it is in fact simpler than Python and Javascript. Immutab…
Common Lisp fan here, but not a data scientist. Why do you say to avoid CL for data analysis? Not trying to flame or anything, just curious about your experience with it.
Clojure, unlike lists in traditional Lisps, based on composable, unified abstraction for its collections, they are lazy by default and literal readable data structures, they are far easier to introspect and not so "opaque" compared to anything - not just CL (even Python), they are superb for dealing with heterogeneous data. Clojure's cohesive data manipulation story is where Common Lisp's lists-and-symbols just can't match.
Re: Python is not a great language for data science
#140> Examples include converting boxplots into violins or vice versa, turning a line plot into a heatmap, plotting a density estimate instead of a histogram, performing a computation on ranked data values instead of raw data values, and so on. Most of this is not about Python, it’s about matplotlib. If you want the admittedly very thoughtful design of ggplot in Python, use plotnine > I would consider the R code to be sl…