221–230 of 339 posts
[1] https://link.springer.com/article/10.1007/s11336-017-9581-x
My experience was that data science was doable but clunky and ugly with pandas. It got slightly better with polars. Only really slightly better. Then, for me at least, it jumped lightyears ahead with duckdb. These days I run some big query on an OLAP database and download the results to parquet stored on the local disk of a cloud notebook VM and then mine it to bits with duckdb reading straight from these parquet fil…
So you're saying you prefer SQL to dataframes. I prefer dataframes and staying in the native language.
I wish people used Julia more. Few years ago I reimplemented some MATLAB code for a novel algorithm [1] I wanted to use in my dissertation about psychometrics and Julia was great language to work with - and also the code ran for 20 minutes instead of 60. [1] https://link.springer.com/article/10.1007/s11336-017-9581-x
Earlier quoted context omitted.
The success of python is due to not needing a broader ecosystem for A LOT of things. They are of course now abandoning this idea.
> The success of python is due to not needing a broader ecosystem for A LOT of things. I honestly think that was a coincidence. Perl and Ruby had other disadvantages, Python won despite having bad package management and a bloated standard library, not because of it.
Most of the python users are not able nor aware of venv, uv, pip and all of that.
I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…
Because they were created by before the need for it and maybe before their invention.
Manipulating numeric arrays and matrices in python is a bit clunky because it was not designed as a scientific computing language so they were added as library. It's much more integrated and natural to use in scientific computer languages such as matlab. However the reverse is also true: because matlab wasn't designed to do what python does, it's a bit clunkier to use outside scientific computing
More terse, more efficient, less error prone, hopefully more numerically accurate, as if Python had an ecosystem of well designed libraries on par with R.
Earlier quoted context omitted.
What is a table other than an array of structs?
I would argue that's about how the data is stored. What I'm trying to express is the idea of the programming language itself supporting high level tabular abstractions/transformations such as grouping, aggregation, joins and so on.
(and yes there's special language support for LINQ so it counts as "part of the language" rather than "a library")