Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python is not a great language for data science

blog.genesmindsmachines.com

Post reply on HN
Print view View original

61–70 of 339 posts

Re: Python is not a great language for data science

#61

Post by dmurray »

mohaine wrote: post #7
dmurray wrote: post #4

Earlier quoted context omitted.

That's not why it's used in data science though. Lots of data scientists use Python all day and have no concept of ever working in a different field. It's used in data science because it's used in data science.

But data science usually isn't an island. Use whatever you want on your one off personal projects but use something more non-data science friendly if you ever want your model to run directly in a production workflow. Productionizing R models is quite painful. The normal way is to just rewrite it not in R.

I've soured a lot on directly productionizing data science code. It's normally an unmaintainable mess.

If you write it in R and then rewrite it in C (better: rewrite it in English with the R as helpful annotations, then have someone else rewrite it in C), at least there is some chance you've thought about the abstractions and operations that are actually necessary for your problem.

Re: Python is not a great language for data science

#62

Post by amai »

The example would better be written in SQL. So according to the author that would make SQL a great language for data science. SQL also supports tables natively. This conclusion is of course ridiculous and shows the shallow reasoning in this article.

Re: Python is not a great language for data science

#63

Post by constantcrying »

RobinL wrote: post #16

I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…

>Why aren't tables first class citizens in programming languages?

Matlab has them, in fact it has multiple competing concepts of it.

Re: Python is not a great language for data science

#64

Post by ActorNightly »

jakobnissen wrote: post #27

Excellent article - except that the author probably should have gated their substantiation of the claim behind a cliffhanger, as other commenters have mentioned. The author's priorities are sensible, and indeed with that set of priorities, it makes sense to end up near R. However, they're not universal among data scientists. I've been a data scientist for eight years, and have found that this kind of plotting and dat…

>Excellent article

Except its not. Data science in python pretty much requires you to use numpy. So his example of mean/variance code is a dumb comparison. Numpy has mean and variance functions built in for arrays.

Even when using raw python in his example, some syntax can be condesed quite a bit:

groups = defaultdict(list) [groups[(row['species'], row['island'])].append(row['body_mass_g']) for row in filtered]

It takes the same amount of mental effort to learn python/numpy as it does with R. The difference is, the former allows you to integrate your code into any other applicaiton.

Re: Python is not a great language for data science

#65

Post by constantcrying »

Python is also an embarrassingly bad language for numerics. It comes without support for different floating point types does not have an n-D Array data type and is extremely slow.

At the same time it is an absolute necessity to know if you are doing numerics. What this shows, at least to me, is that it is "good enough" and that the million integrations, examples and pieces of documentation matter more than whether the peculiarities of the language work in favor of its given use case, as long as the shortcomings can be mostly addressed.

Re: Python is not a great language for data science

#68

Post by p1necone »

RobinL wrote: post #46
CivBase wrote: post #31

Earlier quoted context omitted.

What is a table other than an array of structs?

I would argue that's about how the data is stored. What I'm trying to express is the idea of the programming language itself supporting high level tabular abstractions/transformations such as grouping, aggregation, joins and so on.

Implementing all of those things is an order of magnitude more complex than any other first class primitive datatype in most languages, and there's no obvious "one right way" to do it that would fit everyones use cases - seems like libraries and standalone databases are the way to do it, and that's what we do now.

Re: Python is not a great language for data science

#69

Post by alexnewman »

RobinL wrote: post #16

I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…

APL Is great

Re: Python is not a great language for data science

#70

Post by keeeba »

As a fairly extensive user of both Python and R, I net out similarly.

If I want to wrangle, explore, or visualise data I’ll always reach for R.

If I want to build ML/DL models or work with LLM’s I will usually reach for Python.

Often in the same document - nowadays this is very easy with Quarto.

Post reply on HN