Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python is not a great language for data science

blog.genesmindsmachines.com

Post reply on HN
Print view View original

271–280 of 339 posts

Re: Python is not a great language for data science

#271

Post by [deleted] »

RodgerTheGreat wrote: post #50
RobinL wrote: post #16

I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…

There are a number of dynamic languages to choose from where tables/dataframes are truly first-class datatypes: perhaps most notably Q[0]. There are also emerging languages like Rye[1] or my own Lil[2]. I suspect that in the fullness of time, mainstream languages will eventually fully incorporate tabular programming in much the same way they have slowly absorbed a variety of idioms traditionally seen as part of funct…

[deleted]

Re: Python is not a great language for data science

#272

Post by jstanley »

poulpy123 wrote: post #227
RobinL wrote: post #16

I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…

> Why aren't tables first class citizens in programming languages? Because they were created by before the need for it and maybe before their invention. Manipulating numeric arrays and matrices in python is a bit clunky because it was not designed as a scientific computing language so they were added as library. It's much more integrated and natural to use in scientific computer languages such as matlab. However the…

Tables were definitely around before programming languages.

There are clay tablets from ancient Sumeria that represent information using tables.

Re: Python is not a great language for data science

#273

Post by voidUpdate »

lock1 wrote: post #155
OkayPhysicist wrote: post #130

Earlier quoted context omitted.

Unless you think "most programmers" === "shitty webapp developers", I strongly disagree. Matrices are first class, important components in statistics, data analysis, graphics, video games, scientific computing, simulation, artificial intelligence and so, so much more. And all of those programmers are either using specialized languages, (suffering problems when they want to turn their program into a shitty web app, fo…

That's needlessly aggressive. Ignoring webapps, you could do gamedev without even knowing what a matrix is. You don't even need such construction in most native applications, embedded systems, and OS kernel development.

To be fair, I do use matrices a reasonable amount in gamedev. And if you're writing your engine from scratch, rather than using something like unity, you will almost certainly need matrices

Re: Python is not a great language for data science

#274

Post by hmokiguess »

My issue with Python is that it makes it too easy to do things wrong, it accepts all and anyone. It’s too inclusive and permissive, which is great for expression and creativity but bad for exact sciences and rigid disciplines. In certain matters opinions and cargo cult programming are often a detriment for science. Unfortunately for high level abstractions it’s not that simple to do it right without sacrificing speed, so the industry forces the hand of the community in a lot of ways.

Re: Python is not a great language for data science

#275

Post by tmtvl »

robot-wrangler wrote: post #129
aorist wrote: post #72

> Examples include converting boxplots into violins or vice versa, turning a line plot into a heatmap, plotting a density estimate instead of a histogram, performing a computation on ranked data values instead of raw data values, and so on. Most of this is not about Python, it’s about matplotlib. If you want the admittedly very thoughtful design of ggplot in Python, use plotnine > I would consider the R code to be sl…

>> I would consider the R code to be slightly easier to read (notice how many quotes and brackets the Python code needs) Oh god no, do people write R like that, pipes at the end? Elixir style pipe-operators at the beginning is the way. And if you really wanted to "improve" readability by confusing arguments/functions/vars just to omit quotes, python can do that, you'll just need a wrapper object and getattr hacks to…

The right way is putting the pipe operator at the beginning of the expression.

  (-> (gather-some-data)
    (map 'Vector #'some-functor)
    (filter #'some-predicate)
    (reduce #'some-gatherer))
Or for those who have an irrational fear of brackets:

  ->
    gather-some-data
    map 'Vector #'some-functor
    filter #'some-predicate
    reduce #'some-gatherer

Re: Python is not a great language for data science

#276

Post by nerdponx »

m000 wrote: post #240
jampekka wrote: post #215

Earlier quoted context omitted.

I wonder what the last example of "logistics without libraries" would look like in R. Based on my experience of having to do "low-level" R, it's gonna be a true horror show. In R it's often that things for which there's a ready made libraries and recipes are easy, but when those don't exist, things become extremely hard. And the usual approach is that if something is not easy with a library recipe, it just is not don…

The way you describe it, can we say that R was AI-first without even knowing?

R is overtly and heavily inspired by Lisp which was a big deal in AI at one point. They knew what they were doing.

Re: Python is not a great language for data science

#277

Post by 1gn15 »

orochimaaru wrote: post #270

It’s not. Julia is better, much better. But Julia came too late. A lot of data science code is already in Python. That’s where it’s going to stay because rewriting code is time consuming. My guess is we will continue to improve Python gradually and keep refactoring the code.

> It’s not. Julia is better, much better. But Julia came too late.

Sounds a lot like "worse is better". Python is the worse option, incomplete and inelegant, but is much more practical due to being there first and receiving the bulk of the attention.

Re: Python is not a great language for data science

#279

Post by fithisux »

Personally I use R for the occasional script or some tidyverse quick processing.

But the language has many rough edges

1. non standard eval is very weird, rlang fixes these shortcomings 2. unintuitive names or functions not belonging to packages, base has a mix of functions 3. S3 mixes with naming, no problem personally with S3 and S7 is even better, but mixing S3 names with ordinary names is unintuitive, keep snake case 4. data.frames are unintuitive, tidyverse fixes this 5. f(a=) seriously? or working with unintuitive functions in body for discrete ranges of function arguments? 6. no imports per file in packages, I can live with this .. still ... 7. AST functions are unintuitive

R has some excellent parts:

non-standard evaluation, AST in the base language, lazy evaluation

but it is being killed by the bad parts

I think all the external fixes and sanity in names should go into base

but it will take a lot of time if it ever happens due to legacy.

Julia fixes many of these not as elegantly as R but it's pragmatic approach is too attractive.

Re: Python is not a great language for data science

#280

Post by nyrikki »

> Contrast this with equivalent code that is full of logistics, where I’m using only basic Python language features and no special data wrangling package

While I am not a python cheerleader, but a user because the reality is that it is a pretty good glue language, the above is a bit of a problem.

Duckdb, pandas, numpy etc.. is what makes python nice.

About a decade ago I worked at a major BI software company and ran into another silly problem when trying to evangelize R, wikis kbs and search engines don’t like single letter search terms.

So it didn’t matter how much better R was at the time, people found learning it more difficult than it should have been.

Post reply on HN