Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python is not a great language for data science

blog.genesmindsmachines.com

Post reply on HN
Print view View original

261–270 of 339 posts

Re: Python is not a great language for data science

#261

Post by rkomorn »

freehorse wrote: post #257
rkomorn wrote: post #255

Earlier quoted context omitted.

This happened to me with comprehensions in python, and with JS' love for anonymous/arrow functions. Once you get used to a language's "quirks" (so long as they're considered idiomatic), they no longer feel quirky, and it's usually pretty quick.

You get to the same point with non-considered idiomatic syntax also, the only problem being that it will be only you who understands it.

Only so long as you keep the habit going.

I've definitely written some things that I came back to much later and had to relearn (which is somewhere between embarrassing and humbling).

Re: Python is not a great language for data science

#262

Post by aragilar »

nonameiguess wrote: post #242
m55au wrote: post #136

Earlier quoted context omitted.

Someone correct me if I'm completely wrong, but by default (i.e. precompiled wheels) numpy has 0 dependencies and pandas has 5, one of which is numpy. So not really "squillions" of dependencies. pandas==2.3.3 ├── numpy [required: >=1.22.4, installed: 2.2.6] ├── python-dateutil [required: >=2.8.2, installed: 2.9.0.post0] │ └── six [required: >=1.5, installed: 1.17.0] ├── pytz [required: >=2020.1, installed: 2025.2] └─…

Read https://numpy.org/devdocs/building/blas_lapack.html . NumPy will fall back to internal and very slow BLAS and LAPACK implementations if your system does not have a better one, but assuming you're using NumPy for its performance and not just the convenience of adding array programming features to Python, you're really gonna want better ones, and what that is heavily depends on the computer you're using. This isn'…

The wheels will most likely come with openblas, so while you can get the original blas (which is really only slow by comparison, for small tasks it's likely users won't notice), this is generally not an issue.

Re: Python is not a great language for data science

#263

Post by jakubmazanec »

maratc wrote: post #224
jakubmazanec wrote: post #221

I wish people used Julia more. Few years ago I reimplemented some MATLAB code for a novel algorithm [1] I wanted to use in my dissertation about psychometrics and Julia was great language to work with - and also the code ran for 20 minutes instead of 60. [1] https://link.springer.com/article/10.1007/s11336-017-9581-x

How important was this saving of 40 minutes for the whole timeline of the project of writing your dissertation about psychometrics?

Very important. This was only a simulated dataset, the final analysis would be done on a much larger one (sadly, in the end didn't finish it, because of unrelated reasons). Also, the rewrite didn't take long; the final Julia code was small, few hundreds, or maybe a thousand lines.

Re: Python is not a great language for data science

#264

Post by analog31 »

>>> Without fail, from the students that use Python, the response is: “This will take me a bit. Let me sit down at my desk and figure it out and then I’ll be back.”

This is completely aside, but I wouldn't hold this against the students or Python. The students may be following an age-old rule of office politics: "Never troubleshoot in front of an audience." And why this is more prevalent among the students who use Python, well... sample size of 30.

Re: Python is not a great language for data science

#265

Post by aragilar »

oivey wrote: post #183
shevy-java wrote: post #86

> I think people way over-index Python as the language for data science. It has limitations that I think are quite noteworthy. There are many data-science tasks I’d much rather do in R than in Python. R is kind of a super-specialized language. Python is much more general purpose. R failed to evolve, let's be honest. Python won via jupyter - I see this used ALL the time in universities. R is used too, but mostly for s…

> R failed to evolve, let's be honest. Python won via jupyter Ju = Julia Pyt = Python Er = R R is not only supported in Jupyter, it was there from the start. I’ve never written a single line of R. It is bizarre how little people know about their tools.

But it used to be iPython (and the notebook interface did come out when it was still iPython).

Re: Python is not a great language for data science

#266

Post by aragilar »

vkazanov wrote: post #235
passivegains wrote: post #59

Earlier quoted context omitted.

I think the key thing not obvious to most data scientists is they're not using python because it meets their needs, it's because we've failed them. twice. 1. data scientists aren't programmers, so why do they need a programming language? the tools they should be using don't exist. they'd need programmers to make them, and all we have to offer is... more programming languages. 2. the giant problem at the heart of mode…

When the first scientific libraries were written for python, most alternatives didn't even consider being readable, or convenient. The choice was more like C/Cpp/Fortran vs Python. And then Python went into a self-reinforcing loop, with scientific community coming up with more and more ways to improve Python support for the kind of interactive work that was required for data analysis. Think ipython -> jupyter -> jupy…

I think you're forgetting perl (plus other unix utils) and matlab. PDL (perl data language) was a thing, as was IDL (and other similar tools).

Re: Python is not a great language for data science

#267

Post by kelipso »

IgorPartola wrote: post #256
sgarland wrote: post #156

Earlier quoted context omitted.

Every time I see stuff like this (Google’s new SQL-ish language with pipes comes to mind), I am baffled. SQL to me is eminently readable, and flows beautifully. For reference, I think the same is true of Python, so it’s not like I’m a Perl wizard or something.

Oh I agree. The problem is that they are two different languages. Inside a Python file, SQL is just a string. No syntax highlighting, no compile time checking, etc. A Kwisatz Haderach of languages that incorporates both its own language and SQL as first class concepts would be very nice but the problem is that SQL is just too different. For one thing, SQL is not really meant to be dynamically constructed in SQL. But…

Are you thinking of query generators like Ecto in Elixir?

Re: Python is not a great language for data science

#268

Post by gbacon »

lenkite wrote: post #198
Surac wrote: post #194

I at the moment try to learn python as a hobby language. I use c c++ and c# to earn my money. MY biggest problem is finding good examples that are up to date. I spent a whole day learning that there a four (I think) ways to do formatting strings. This „bloat“ in syntax makes even a simple print very heavy to digest. I don’t even bother using v2 python only v3. Also using whitespaces to block things together sounds ap…

15 years ago, Python programmers used to mock Perl by quoting the Zen of Python: "There should be one - and preferably only one - obvious way to do it.". This was in stark contrast to Perl's TIMTOWTDI motto: "There Is More Than One Way To Do It." The Zen of Python is sadly now an absolute lie.

Rather than an absolute lie, I’m more inclined to characterize it as naïve or black-and-white thinking, outside of CRUD apps and undergraduate intro projects.

Re: Python is not a great language for data science

#269

Post by Vaslo »

My team has all moved slowly from R to Python. There was no pressure to do so. R has a clunky feel with a bunch of modules that can be a challenge to automate. Python’s general purpose use beats whatever superior modules R has all day. If someone wants the same package on Python from R it’s probably out there.

While plotting may be clunky, I just don’t see r as much better. Plus in 2025 I can just provide a sample of data and what plot I want in an LLM and I get zero shot code of the plot I want.

Author sounds very academic to me.

Re: Python is not a great language for data science

#270

Post by orochimaaru »

It’s not. Julia is better, much better. But Julia came too late.

A lot of data science code is already in Python. That’s where it’s going to stay because rewriting code is time consuming. My guess is we will continue to improve Python gradually and keep refactoring the code.

Post reply on HN