Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python is not a great language for data science

blog.genesmindsmachines.com

Post reply on HN
Print view View original

161–170 of 339 posts

Re: Python is not a great language for data science

#161

Post by getnormality »

plaidfuji wrote: post #152

Python is a pretty bad language for tabular data analysis and plotting, which seems to be the actual topic of this post. R is certainly better, hell Tableau, Matlab, JMP, Prism and even Excel are all better in many cases. Pandas+seaborn has done a lot, but seaborn still has frustrating limits. And pandas is essentially a separate programming language. If your data is already in a table, and you’re using Python, you’r…

...unless your data engineering job happens on a database, in which case R's dbplyr is far better than anything Python has to offer.

Re: Python is not a great language for data science

#163

Post by [deleted] »

RobinL wrote: post #16

I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…

[deleted]

Re: Python is not a great language for data science

#164

Post by 7thaccount »

iLemming wrote: post #134
7thaccount wrote: post #122

Earlier quoted context omitted.

I tried to get into Clojure, but a lot of the JVM hosted languages require some Java experience. Same thing with Scala and Kotlin or F# on .NET. The early tooling was also pretty dependent on Vim or Emacs. Maybe it's all easier now with VSCode or something like that.

None of this even remotely true. I've gotten into Clojure without knowing jackshit about Java, almost ten years later, after tons of things successfully built and deployed, still don't know jackshit about Java. Mia, co-host of 'Clojure apropos' podcast was my colleague, we've worked together on multiple teams, she learned Clojure as her very first PL. Later she tried learning some Java and she was shocked how impossi…

It might not be a problem for you, but it has been for many. I did start by reading through 3 Clojure books. The repl and the basic stuff like using lists is all easy of course, but the tooling was pretty poor compared to what I was used to (I like lisp, but Emacs is a commitment). Also, a lot of tutorials at the time definitely assumed java familiarity, especially with debugging java stack traces.

Re: Python is not a great language for data science

#165

Post by 3eb7988a1663 »

_yb2s wrote: post #145

Doing computational biology for several decades in about a dozen languages, I do think R is a much better language for data science, but in practice I end up using Python almost every time because it has more libraries, and it’s easier to find software engineers and collaborators to work on Python. However, R makes for much simpler cleaner code, less silent errors, and the 1 indexing makes dealing with biological seq…

Pardon? Less silent errors? R has quite a few foot guns around permissively parsing user intention. Which does make it handy for exploratory analysis, but a lot more fragile when you want production code.

Just a simple one that can get you, R is 1-indexed. Yet if you have a vector, accessing myvec[0] is not an error. Alternatively, if you had say, a vector length of 3 and do myvec[10] that gets NA (an otherwise legal value). Or you could make an assignment past the end of the vector myvec[15] <- 3.14 , which will silently extend the array, inserting NAs

Re: Python is not a great language for data science

#166

Post by Iwan-Zotow »

getnormality wrote: post #159
RobinL wrote: post #16

I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…

Saying that SQL is the standard for manipulating tabular data is like saying that COBOL is the standard for financial transactions. It may be true based on current usage, but nobody thinks it's a good idea long term. They're both based on the outdated idea that a programming language should look like pidgin English rather than math.

In R data.table is basically SQL in another shape

Re: Python is not a great language for data science

#167

Post by Iwan-Zotow »

nextos wrote: post #41
RobinL wrote: post #16

I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…

I don't think this is the real problem. In R and Julia tables are great, and they are libraries. The key is that these languages are very expressive and malleable. Simplifying a lot, R is heavily inspired by Scheme, with some lazy evaluation added on top. Julia is another take at the design space first explored by Dylan.

R was clone of S

Re: Python is not a great language for data science

#168

Post by jltsiren »

OkayPhysicist wrote: post #120
RobinL wrote: post #16

I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…

There's a number of structures that I think are missing in our major programming languages. Tables are one. Matrices are another. Graphs, and relatedly, state machines are tools that are grossly underused because of bad language-level support. Finally, not a structure per se, but I think most languages that are batteries-included enough to included a regex engine should have a a full-fledged PEG parsing engines. Most…

When there is no clear canonical way of implementing something, adding it to a programming language (or a standard library) is risky. All too often, you realize too late that you made a wrong choice, and then you add a second version. And a third. And so on. And then you end up with a confusing language full of newbie traps.

Graphs are a good example, as they are a large family of related structures. For example, are the edges undirected, directed, or something more exotic? Do the nodes/edges have identifiers and/or labels? Are all nodes/edges of the same type, or are there multiple types? Can you have duplicate edges between the same nodes? Does that depend on the types of the nodes/edges, or on the labels?

Re: Python is not a great language for data science

#169

Post by liveranga »

RodgerTheGreat wrote: post #50
RobinL wrote: post #16

I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…

There are a number of dynamic languages to choose from where tables/dataframes are truly first-class datatypes: perhaps most notably Q[0]. There are also emerging languages like Rye[1] or my own Lil[2]. I suspect that in the fullness of time, mainstream languages will eventually fully incorporate tabular programming in much the same way they have slowly absorbed a variety of idioms traditionally seen as part of funct…

Nushell is another one with tables built-in:

https://www.nushell.sh/book/working_with_tables.html

Re: Python is not a great language for data science

#170

Post by kelipso »

pphysch wrote: post #128
Joel_Mckay wrote: post #78

Earlier quoted context omitted.

Python has a list of issues fundamentally broken in the language, and relies heavily on integrated library bindings to operate at reasonable speeds/accuracy. Julia allows embedding both R and Python code, and has some very nice tools for drilling down into datasets: https://www.queryverse.org/ It is the first language I've seen in decades that reduces entire paradigms into single character syntax, often outperforming…

Deeply ironic for a Julia proponent to smear a popular language as "fundamentally broken" without evidence. https://yuri.is/not-julia/

This is like one of those people posting Dijkstra’s letter advocating for 0-based indexing without ever having read or understood what they posted.
Post reply on HN