okay, click bait worked on me. but the claims are weak. basically "Python is not a great language... because it's not that of a domain language than R"
mediocre!
Python is not a great language for data science
311–320 of 339 posts
Re: Python is not a great language for data science
#312My experience was that data science was doable but clunky and ugly with pandas. It got slightly better with polars. Only really slightly better. Then, for me at least, it jumped lightyears ahead with duckdb. These days I run some big query on an OLAP database and download the results to parquet stored on the local disk of a cloud notebook VM and then mine it to bits with duckdb reading straight from these parquet fil…
Huh, as a frequent polars user, I'll try duckdb.
well, duckdb works very well with pandas, too
Re: Python is not a great language for data science
#313I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…
There's a number of structures that I think are missing in our major programming languages. Tables are one. Matrices are another. Graphs, and relatedly, state machines are tools that are grossly underused because of bad language-level support. Finally, not a structure per se, but I think most languages that are batteries-included enough to included a regex engine should have a a full-fledged PEG parsing engines. Most…
Would love to see a language in which hierarchical state machines, math/linear algebra, I/O to sensors and actuators, and time/timing were first class citizens.
Mainly for programming control systems for robotics and aerospace applications
Re: Python is not a great language for data science
#314Earlier quoted context omitted.
Well, sure, but equally I think there would have been a lot more effort to fix the packaging nightmare if it had been more urgent.
There was a massive effort though, the proliferation of several different package managers is evidence of that.
Maybe. A lot of them felt like one-person projects that not many people cared about. I think that on the contrary, part of the reason so many different package managers could coexist with no clear winner emerging was that the problem wasn't very serious for a lot of the community.
Re: Python is not a great language for data science
#315Kotlin is trying to be one with notebooks[0], I even heard they have fancy code generation so that your dynamic data can still have typed properties (after the first evaluations, members corresponding to your field names are generated, or something to that extent I never used it)
[0] https://kotlinlang.org/docs/kotlin-notebook-overview.html
Re: Python is not a great language for data science
#316This was underwhelming. I work with Python and Pandas, and I can show examples of much clumsier workflows I run into. The most often, you get dataframe[(dataframe.column1 == something) & ~dataframe.column2.isna()] constucts, which show that python syntax falls short here, and isn't suitable for such manipulations. Unfortunately, there's no alternative, and I don't see R as much easier, there are plenty of ugly things…
> Unfortunately, there's no alternative, and I don't see R as much easier, there are plenty of ugly things as well there. Have you tried Polars? It really discourages the inefficient creation of intermediate boolean arrays such as in the code that you are showing. > There's Julia -- it has serious drawbacks, like slow cold start if you launch a Julia script from the shell, which makes it unsuitable for CLI workflows.…
Yes, I tried Polars both in Python and Rust. In Python it's more verbose, as the other commenter points out. In Rust it's very painful.
Re: Python is not a great language for data science
#317Earlier quoted context omitted.
Unless you think "most programmers" === "shitty webapp developers", I strongly disagree. Matrices are first class, important components in statistics, data analysis, graphics, video games, scientific computing, simulation, artificial intelligence and so, so much more. And all of those programmers are either using specialized languages, (suffering problems when they want to turn their program into a shitty web app, fo…
That's needlessly aggressive. Ignoring webapps, you could do gamedev without even knowing what a matrix is. You don't even need such construction in most native applications, embedded systems, and OS kernel development.
I really doubt this.
Even through UE blueprints (assuming the most high level abstraction here) you will come across the need to perform calculations with matrices. While a lot is abstracted away, you still need to know about coordinate spaces, quirks around order of operations, etc.
Re: Python is not a great language for data science
#318Earlier quoted context omitted.
That's needlessly aggressive. Ignoring webapps, you could do gamedev without even knowing what a matrix is. You don't even need such construction in most native applications, embedded systems, and OS kernel development.
To be fair, I do use matrices a reasonable amount in gamedev. And if you're writing your engine from scratch, rather than using something like unity, you will almost certainly need matrices
Even when using some game engine you need some prior knowledge. Otherwise you are just throwing stuff at the wall and see if it sticks.
Re: Python is not a great language for data science
#319I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…
I'd say there are converging standards like Parquet for longterm on disk, Arrow for in memory cross language, and increasingly duckdb for just standard SQL on that in memory or on disk representation. If I had to guess most of the data table things vanish long term because everyone can just use SQL now for all the stuff they did with quirky hacked up APIs and patchy performance because of those hacked up APIs.
Re: Python is not a great language for data science
#320Earlier quoted context omitted.
Yes, R is a proper general purpose programming language. Turing complete, functional, procedural, object oriented.../
Just in case someone reads this far and sees blubber's confident "No." Blubber is definitely wrong here. I used to do all of my programming in R. Throw the question into an LLM if you're wondering if R has a package like ___ in python.
I know people who used Visual Basic for all of their programming. I'd say No either way unless people explained to me without bursting out into laughter that they also have extensive experience with, e.g., Kotlin, Rust, C#, Java etc. and still prefer VB or R for non-trivial programs.