Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python is not a great language for data science

blog.genesmindsmachines.com

Post reply on HN
Print view View original

331–339 of 339 posts

Re: Python is not a great language for data science

#331

Post by IgorPartola »

kelipso wrote: post #267
IgorPartola wrote: post #256

Earlier quoted context omitted.

Oh I agree. The problem is that they are two different languages. Inside a Python file, SQL is just a string. No syntax highlighting, no compile time checking, etc. A Kwisatz Haderach of languages that incorporates both its own language and SQL as first class concepts would be very nice but the problem is that SQL is just too different. For one thing, SQL is not really meant to be dynamically constructed in SQL. But…

Are you thinking of query generators like Ecto in Elixir?

I am not familiar with them but when I think of query generators I think of the lower level API for SQLAlchemy which is fine but still kludgy as it tries to translate SQL into a new “language” that is less known and less intuitive and still requires you to think in terms of is the data you are working with local or remote.

This is why key-value stores are so popular, I think. They make you do more but with all local data (that is data in your memory not in the database server). SQL can do a lot but because we almost never represent a user object as just a tuple there is a fundamental impedance mismatch between an environment that only deals with tuples in tables and an environment that deals with objects of some kind. Something that can do both at once would be the ultimate. Maybe the way to look at it isn’t to bring the database into your application but to run the entirety of the application inside a database. Imagine if all your business logic could easily be encoded into stored procedures and all you had to do was expose endpoints to draw a UI for it. That might actually work (and I know there are some systems that try this but none are mainstream enough).

Re: Python is not a great language for data science

#332

Post by antononcube »

SubjectToChange wrote: post #282
RobinL wrote: post #16

I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…

Mathematica recently added the Tabular command, for what it’s worth. I haven’t used it much yet, but it seems to be quite capable.

Yes, Wolfram Language (WL) -- aka Mathematica -- introduced `Tabular` in 2025. It is a new data structure with a constellation of related functions (like `ToTabular`, `PivotToColumns`, etc.) Using it is 10÷100 times faster than using WL's older `Dataset` structure. (In my experience. With both didactic and real life data of 1_000÷100_000 rows and 10÷100 columns.)

Re: Python is not a great language for data science

#333

Post by northlondoner »

blubber wrote: post #320
steine65 wrote: post #304

Earlier quoted context omitted.

Just in case someone reads this far and sees blubber's confident "No." Blubber is definitely wrong here. I used to do all of my programming in R. Throw the question into an LLM if you're wondering if R has a package like ___ in python.

I know people who used Visual Basic for all of their programming. I'd say No either way unless people explained to me without bursting out into laughter that they also have extensive experience with, e.g., Kotlin, Rust, C#, Java etc. and still prefer VB or R for non-trivial programs.

blubber, I think there might be some misconceptions. Just for the record.

R is not actually competing with those languages. R's design purpose is different. it is a general purpose computational language for scientists. There are FFIs (Foreign Function Interfaces) for all those languages.

R-Kotlin-Jave: https://journal.r-project.org/articles/RJ-2018-066/ R-Rust: https://cran.r-project.org/web/packages/using_rust.html R-C# : https://github.com/Open-Systems-Pharmacology/rsharp/

R is supporting C integration natively anyhow (see Chambers's book.

Regarding VB reference. VB was used in finance a lot to do some advanced maths. just a side remark.

Re: Python is not a great language for data science

#334

Post by fastasucan »

willvarfar wrote: post #322
mettamage wrote: post #214

Earlier quoted context omitted.

Huh, as a frequent polars user, I'll try duckdb.

I think you can query and join polars dataframes if you call to_arrow() on them too.

You dont need to do that afaik. You just query them.

Re: Python is not a great language for data science

#335

Post by fastasucan »

plaidfuji wrote: post #152

Python is a pretty bad language for tabular data analysis and plotting, which seems to be the actual topic of this post. R is certainly better, hell Tableau, Matlab, JMP, Prism and even Excel are all better in many cases. Pandas+seaborn has done a lot, but seaborn still has frustrating limits. And pandas is essentially a separate programming language. If your data is already in a table, and you’re using Python, you’r…

> And pandas is essentially a separate programming language.

No it isn't.

Re: Python is not a great language for data science

#337

Post by blubber »

northlondoner wrote: post #321
blubber wrote: post #320

Earlier quoted context omitted.

I know people who used Visual Basic for all of their programming. I'd say No either way unless people explained to me without bursting out into laughter that they also have extensive experience with, e.g., Kotlin, Rust, C#, Java etc. and still prefer VB or R for non-trivial programs.

Of course R isn't a complied language and probably not the same category as C/Rust as systems language but is not in the same category as VB. R is a serious scientific programming language used in non-trivial programs for industrial applications. See Posit's customers. I suggest John Chambers ( https://en.wikipedia.org/wiki/John_Chambers_(statistician) ) book, he explain how he designed S language, R's grandfather so…

This isn't about compilation vs interpretation. R is simply badly designed as a programming language. This doesn't change just because its inventor wrote a book.

Re: Python is not a great language for data science

#338

Post by hekkle »

fnord77 wrote: post #325
hekkle wrote: post #102

Earlier quoted context omitted.

Not great at what? I agree that Python is not great at anything specifically, but it is good at almost everything, and that's what makes it great.

which is nicer? float(len(str)) or str.len().float() Also, there's numerous ways of doing the same thing, despite that being contrary to python's philosophy

Actchooally: Python's philosophy (the zen of python) states:

There should be one-- and preferably only one --obvious way to do it.

So, the answer is to pick the obvious choice, which in this case will depend on the style guide of the project you are working on, and whether they prefer the lisp style syntax or the OO property of style.

Post reply on HN