Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python is not a great language for data science

blog.genesmindsmachines.com

Post reply on HN
Print view View original

181–190 of 339 posts

Re: Python is not a great language for data science

#181

Post by geokon »

7thaccount wrote: post #122
iLemming wrote: post #58

Earlier quoted context omitted.

I can't speak for Julia - never used it; never used Common Lisp for analyzing data (I don't think it's very "data-oriented" for the modern age and the shape of data), but Clojure is really not "obscure" - it only looks weird for the first fifteen minutes or so; once you start using it - it is one of the most straightforward and reasonable languages out there - it is in fact simpler than Python and Javascript. Immutab…

I tried to get into Clojure, but a lot of the JVM hosted languages require some Java experience. Same thing with Scala and Kotlin or F# on .NET. The early tooling was also pretty dependent on Vim or Emacs. Maybe it's all easier now with VSCode or something like that.

It doesn't require any Java but the docs do at times sort of assume you understand the JVM to some extent - which was a bit frustrating when first learning the language. It'll use terms like "classpath" without explaining what that is. However nowadays with LLMs these are insignificant speedbumps.

If you want to use Java you also don't really need to know Java beyond "you create instances of classes and call methods on them". I really don't want to learn a dinosaur like Java, but having access to the universe of Java libs has saved me many times. It's super fun and nice to use and poke around mature Java libs interactively with a REPL :)

All that said I'd have no idea how to write even a helloworld in Java

PS: Agreed on Emacs. I love Emacs.. but it's for turbo nerds. Having to learn Emacs and Clojure in parallel was a crazy barrier. (and no, Emacs is not as easy people make it out to be)

Re: Python is not a great language for data science

#182

Post by geokon »

phforms wrote: post #52
iLemming wrote: post #8

From many practical points, Clojure is great for data. And you can even leverage python libs via clj-python.

In the past few years I have seen some serious efforts from the Clojure community to make Clojure more attractive for data science. Check out the Scicloj[1] group and their data science stack/toolkit Noj[2] (still in beta) as well as the high-performance tabular data processing library tech.ml.dataset (TMD)[3]. - [1] https://scicloj.github.io - [2] https://scicloj.github.io/noj - [3] https://github.com/techascent/tec…

What's worth emphasizing is that you're not marrying in to an ecosystem of libs. There are a lot of separate pieces that you can typically use separately. I do climate data work without most of Scicloj's tools, but I do use tech.ml.dataset extensively

Re: Python is not a great language for data science

#183

Post by oivey »

shevy-java wrote: post #86

> I think people way over-index Python as the language for data science. It has limitations that I think are quite noteworthy. There are many data-science tasks I’d much rather do in R than in Python. R is kind of a super-specialized language. Python is much more general purpose. R failed to evolve, let's be honest. Python won via jupyter - I see this used ALL the time in universities. R is used too, but mostly for s…

> R failed to evolve, let's be honest. Python won via jupyter

Ju = Julia Pyt = Python Er = R

R is not only supported in Jupyter, it was there from the start. I’ve never written a single line of R. It is bizarre how little people know about their tools.

Re: Python is not a great language for data science

#184

Post by noitpmeder »

m55au wrote: post #136
throwaway2037 wrote: post #125

Earlier quoted context omitted.

I hear this so much from Python people -- almost like they are paid by the word to say it. Is it different from Perl, Ruby, Java, or C# (DotNet)? Not in my experience, except people from those communities don't repeat that phrase so much. The irony here: We are talking about data science. 98% of "data science" Python projects start by creating a virtual env and adding Pandas and NumPy which have numerous (really: squ…

Someone correct me if I'm completely wrong, but by default (i.e. precompiled wheels) numpy has 0 dependencies and pandas has 5, one of which is numpy. So not really "squillions" of dependencies. pandas==2.3.3 ├── numpy [required: >=1.22.4, installed: 2.2.6] ├── python-dateutil [required: >=2.8.2, installed: 2.9.0.post0] │ └── six [required: >=1.5, installed: 1.17.0] ├── pytz [required: >=2020.1, installed: 2025.2] └─…

I don't know about _squillions_, but numpy definitely has _requirements_, even if they're not represented as such in the python graph.

e.g.

  https://github.com/numpy/numpy/blob/main/.gitmodules (some source code requirements)
  https://github.com/numpy/numpy/tree/main/requirements (mostly build/ci/... requirements)
  ...

Re: Python is not a great language for data science

#186

Post by don-bright »

RobinL wrote: post #16

I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…

Every copy of Microsoft Excel includes Power Query which is in the M language and has tables as a type. Programs are essentially transformations of table columns and rows. Not sure if its mainstream but is widely available. M language is also included in other tools like PowerBI and Power Automate.

Re: Python is not a great language for data science

#187

Post by m55au »

noitpmeder wrote: post #184
m55au wrote: post #136

Earlier quoted context omitted.

Someone correct me if I'm completely wrong, but by default (i.e. precompiled wheels) numpy has 0 dependencies and pandas has 5, one of which is numpy. So not really "squillions" of dependencies. pandas==2.3.3 ├── numpy [required: >=1.22.4, installed: 2.2.6] ├── python-dateutil [required: >=2.8.2, installed: 2.9.0.post0] │ └── six [required: >=1.5, installed: 1.17.0] ├── pytz [required: >=2020.1, installed: 2025.2] └─…

I don't know about _squillions_, but numpy definitely has _requirements_, even if they're not represented as such in the python graph. e.g. https://github.com/numpy/numpy/blob/main/.gitmodules (some source code requirements) https://github.com/numpy/numpy/tree/main/requirements (mostly build/ci/... requirements) ...

They're not represented, because those are build-time dependencies. Most users when they do pip install numpy or equivalent, just get the precompiled binaries and none of those get installed. And even if you compile it yourself, you still don't need those for running numpy.

Re: Python is not a great language for data science

#188

Post by gyulai »

I think, the lesson learned from › Python v. R ‹ is that people prefer doing data science in a general purpose language that is also okay-ish for data science over a language that's purpose-built for data science but suffers from diseconomies. Specifically: Imagine a new database or something like that has just come out. Now, the audience that wants to wire it into applications and the audience that wants to tap it to extract data for analytics put their weight together to create the demand for the Python library. The economies for that work out better than if you had to create two different libraries in two different languages to satisfy those two groups of demand.

Re: Python is not a great language for data science

#189

Post by jillesvangurp »

niemandhier wrote: post #38

Python is just a language that: 1. Is easy to read 2. Was easy to extend in languages that people who work with scientific data happen to like. When I did my masters we hacked around in the numpy source and contributed here and there while doing astrophysics. Stuff existed in Java and R, but we had learned C in the first semester and python was easier to read and contrary to MATLAB numpy did not need a license. When…

The main feature of Python is that it is approachable by people who have never programmed before. They might have a vague notion of wanting to instruct a computer to first do this and then do that. Imperative programming is their starting point. And Python delivers that. It was designed as a scripting language whose primary use indeed was to script together other things. It always was good at that and that was the main thing it was used for in the nineties.

It got popular once Linux distributions started relying on a lot of python scripts (e.g. Red Hat and Debian). As a side effect it was present on a lot of Linux and Unix systems early on. Scientists in the early 2000s and late nineties had access to workstations running Linux and Unix. So, Python was simply the approachable thing that was just there already.

And because it's so easy, there are lots of people getting into Python. So it got its own dynamic of generations of researchers in all sorts of fields knowing about Python being the goto thing to reach for. It never really was the best at anything it does. That wasn't even a goal. It's a bit slow. A bit verbose/clumsy compared to some of the alternatives that some data scientists prefer. It lacks a lot of features other languages have. Etc. This doesn't matter because it is simple and easy. The type of users that are new to programming are looking for something simple that they can understand. Not the platonic ideal of a language that mathematicians or computer scientists might prefer.

Python is the modern equivalent of BASIC which had this role before python was created. It wasn't that amazing. But early home computers had it as part of their OS. E.g. the Commodore 64 that was my first computer had an interactive Basic shell with the ability to load games from a tape as the main OS experience.

Re: Python is not a great language for data science

#190

Post by LanceH »

gyulai wrote: post #188

I think, the lesson learned from › Python v. R ‹ is that people prefer doing data science in a general purpose language that is also okay-ish for data science over a language that's purpose-built for data science but suffers from diseconomies. Specifically: Imagine a new database or something like that has just come out. Now, the audience that wants to wire it into applications and the audience that wants to tap it t…

You mention a good point of using Python to put out the results.

I think munging the input into a clean enough data set that you can work on is another place Python excels compared to analysis specific tools like R.

Post reply on HN