Python is not a great language for data science • Hacker News Board

Re: Python is not a great language for data science

#231

Post by HelloNurse » Wed, Nov 26, 2025, 9:40 AM UTC

OkayPhysicist wrote: post #120

RobinL wrote: post #16

I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…

There's a number of structures that I think are missing in our major programming languages. Tables are one. Matrices are another. Graphs, and relatedly, state machines are tools that are grossly underused because of bad language-level support. Finally, not a structure per se, but I think most languages that are batteries-included enough to included a regex engine should have a a full-fledged PEG parsing engines. Most…

> I think most languages that are batteries-included enough to included a regex engine should have a a full-fledged PEG parsing engines

Then there would be more PEG horror stories. In addition, string and indices in regex processing are universal, while a parser is necessarily more framework-like, far more complex and doomed to be mismatched for many applications.

Re: Python is not a great language for data science

#232

Post by Joel_Mckay » Wed, Nov 26, 2025, 9:40 AM UTC

pphysch wrote: post #177

kelipso wrote: post #170

Earlier quoted context omitted.

This is like one of those people posting Dijkstra’s letter advocating for 0-based indexing without ever having read or understood what they posted.

What does indexing syntax have to do with Julia having a rough history of correctness bugs and footguns?

Sure, all software is terrible if looking at bug frequency history...

https://github.com/python/cpython/issues

Griefers ranting about years old _closed_ tickets on v1.0.5 versions on a blog as some sort of proof of lameness... is a poorly structured argument. Julia includes regression testing features built into even its plotting library output, and thus issues usually stay resolved due to pedantic reproducibility. Also, running sanity-checks in any llvm language code is usually wise.

Best of luck =3

Re: Python is not a great language for data science

#233

Post by poulpy123 » Wed, Nov 26, 2025, 9:41 AM UTC

But python is a great language for data science. As the anglos say: the proof is in the pudding, and the fact it is massively used for data science prove it is great at data science.

You will say that not everything that is successful is great, and you will be right, but the success of python came organically, and not because of advertisement, de facto monopoly, politics, money, or first-arrived-advantage.

Although there is one cause that isn't intrinsic to python but from the people who built numpy. The fact there is a single numerical library, extremely easy to use, fast and extensive in the whole ecosystem was very very huge

Re: Python is not a great language for data science

#234

Post by Pinegulf » Wed, Nov 26, 2025, 9:52 AM UTC

Once the data is clean and neatly in standard format this becomes a matter of preference.

Work experience says that 90% of work is gathering, cleaning and transforming data from different sources. In this capacity Python has more options available.

Re: Python is not a great language for data science

#235

Post by vkazanov » Wed, Nov 26, 2025, 9:54 AM UTC

passivegains wrote: post #59

vkazanov wrote: post #11

Earlier quoted context omitted.

It's used in data science because no other language has this level of library support. And it got this unprecedented level of support because right from the start it made its focus clear syntax and (perceived) simplicity. There is also a sort of cumulative effect from being nice for algorithmic work. Guido's long-term strategy won over numerous other strong candidates for this role.

I think the key thing not obvious to most data scientists is they're not using python because it meets their needs, it's because we've failed them. twice. 1. data scientists aren't programmers, so why do they need a programming language? the tools they should be using don't exist. they'd need programmers to make them, and all we have to offer is... more programming languages. 2. the giant problem at the heart of mode…

When the first scientific libraries were written for python, most alternatives didn't even consider being readable, or convenient. The choice was more like C/Cpp/Fortran vs Python.

And then Python went into a self-reinforcing loop, with scientific community coming up with more and more ways to improve Python support for the kind of interactive work that was required for data analysis. Think ipython -> jupyter -> jupyter forks and other python-centric notebook systems.

So when data analysis evolved into data science and machine learning, gpu-first library vendors already faced a crowd of people knowing python.

It is crazy how right now one can utilize 100s of gpus through these bits of dirty python wrapped in json.

Re: Python is not a great language for data science

#236

Post by vkazanov » Wed, Nov 26, 2025, 9:58 AM UTC

rjzzleep wrote: post #193

lmm wrote: post #157

Earlier quoted context omitted.

> The success of python is due to not needing a broader ecosystem for A LOT of things. I honestly think that was a coincidence. Perl and Ruby had other disadvantages, Python won despite having bad package management and a bloated standard library, not because of it.

It's because Ruby captured the web market and Python everything else, and I get everything is more timeless than a single segment.

Ruby was competing on the web market and lost to many others, including Python. In part, because python had a much broader ecosystem, and php had wide adoption through wordpress and others, and javascript was expanding from browsers.

Re: Python is not a great language for data science

#237

Post by markkitti » Wed, Nov 26, 2025, 10:03 AM UTC

I tried this in Julia with TidierData.jl, and it looks quite similar to the R version.

  using TidierData, DataFrames
  using PalmerPenguins: load

  penguins = load()

  @chain penguins begin
    DataFrame
    @drop_missing(body_mass_g)
    @group_by(species, island)
    @summarize(
      body_weight_mean =
        mean(body_mass_g),
      body_weight_std =
        std(body_mass_g)
    )
    show(_, allrows=true)
  end

Re: Python is not a great language for data science

#238

Post by HenriTEL » Wed, Nov 26, 2025, 10:25 AM UTC

RobinL wrote: post #16

I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…

Well you nailed it, the language you're looking for is SQL. There's a reason why duckdb got such traction over the last years. I think data scientists overlook SQL and Excel like tooling.

Re: Python is not a great language for data science

#239

Post by blubber » Wed, Nov 26, 2025, 10:38 AM UTC

aorist wrote: post #72

> Examples include converting boxplots into violins or vice versa, turning a line plot into a heatmap, plotting a density estimate instead of a histogram, performing a computation on ranked data values instead of raw data values, and so on. Most of this is not about Python, it’s about matplotlib. If you want the admittedly very thoughtful design of ggplot in Python, use plotnine > I would consider the R code to be sl…

"The reason you can use this simpler syntax in R is because it’s non-standard-evaluation ..."

So it actually is about Python vs R.

That said, while this kind of non-standard evaluation is nice when working interactively on the command line, I don't think it's that relevant when writing code for more elaborated analyses. In that context, I'd actually see this as a disadvantage of R because you suddenly have to jump through loops to make trivial things work with that non-standard evaluation.

Re: Python is not a great language for data science

#240

Post by m000 » Wed, Nov 26, 2025, 10:39 AM UTC

jampekka wrote: post #215

aorist wrote: post #72

> Examples include converting boxplots into violins or vice versa, turning a line plot into a heatmap, plotting a density estimate instead of a histogram, performing a computation on ranked data values instead of raw data values, and so on. Most of this is not about Python, it’s about matplotlib. If you want the admittedly very thoughtful design of ggplot in Python, use plotnine > I would consider the R code to be sl…

I wonder what the last example of "logistics without libraries" would look like in R. Based on my experience of having to do "low-level" R, it's gonna be a true horror show. In R it's often that things for which there's a ready made libraries and recipes are easy, but when those don't exist, things become extremely hard. And the usual approach is that if something is not easy with a library recipe, it just is not don…

The way you describe it, can we say that R was AI-first without even knowing?