Python is also an embarrassingly bad language for numerics. It comes without support for different floating point types does not have an n-D Array data type and is extremely slow. At the same time it is an absolute necessity to know if you are doing numerics. What this shows, at least to me, is that it is "good enough" and that the million integrations, examples and pieces of documentation matter more than whether th…
Native python is hopeless for numerics, which is why just about everyone just uses numpy, which solves all of these issues. Of course, a separate package. But the strength of python is that it can fairly seamlessly incorporate these kinds of packages that add core capabilities. Another important example: pytorch.
Python is not a great language for data science
291–300 of 339 posts
Re: Python is not a great language for data science
#292Python pays the bills. If it was up to me I'd use a different language, but there is no denying that its got a strong story in just about every field now. As I've gotten older, I've come to realize that programming languages are vehicles for solving computer based problems, and I've learned to find joy in solving those problems in whatever language my company/project is using.
But in my personal projects, my favorite language to use it Dart.
Re: Python is not a great language for data science
#293Doesn’t need to be great, just needs to be good enough.
Re: Python is not a great language for data science
#294Earlier quoted context omitted.
There's a number of structures that I think are missing in our major programming languages. Tables are one. Matrices are another. Graphs, and relatedly, state machines are tools that are grossly underused because of bad language-level support. Finally, not a structure per se, but I think most languages that are batteries-included enough to included a regex engine should have a a full-fledged PEG parsing engines. Most…
When there is no clear canonical way of implementing something, adding it to a programming language (or a standard library) is risky. All too often, you realize too late that you made a wrong choice, and then you add a second version. And a third. And so on. And then you end up with a confusing language full of newbie traps. Graphs are a good example, as they are a large family of related structures. For example, are…
Even the raw storage for graphs doesn't have just one answer: you could store edge lists or you could store adjacency matrixes. Some algorithms work better with one, some work better with the other. You probably don't want to store both because that can be extra memory overhead as well as a locking problem if you need to atomically update both at once. You probably don't want to automatically flip back and forth between representations because that could cause garbage collector churn if not also long breadth or depth searches, and you may not want to encourage manual conversions between data structures either (to avoid providing a performance footgun to your users). So you probably want the edge list Graph type and the adjacency matrix Graph type to look very different, even though (they are trivially convertible they may be expensive to convert as mentioned), and yeah that's the under-the-hood storage mechanism. From there you get into possible exponential explosion as you start to get into the other higher level distinctions between types of graphs (DAGs versus Trees versus cyclic structures and so forth, and all the variations on what a node can be, if edges can be weighted or labeled, etc).
Re: Python is not a great language for data science
#295Earlier quoted context omitted.
Sure, all software is terrible if looking at bug frequency history... https://github.com/python/cpython/issues Griefers ranting about years old _closed_ tickets on v1.0.5 versions on a blog as some sort of proof of lameness... is a poorly structured argument. Julia includes regression testing features built into even its plotting library output, and thus issues usually stay resolved due to pedantic reproducibility. A…
Just saying, "other languages have bug reports" is a exceptionally poor way to promote Julia =3
To be blunt: Moores law is now effectively dead, and chasing the monolithic philosophy with lazy monads will eventually limit your options.
Languages like Julia trivially handle conditional parallelism much more cleanly with the broadcast operator, and transparent remote host process instancing over ssh (still needs a lot of work to reach OTP like cluster functionality.)
Much like Go, library resources ported into the native language quietly moves devs away from the same polyglot issues that hit Python.
Best of luck. =3
Re: Python is not a great language for data science
#296I think a lot of this comes down to the question: Why aren't tables first class citizens in programming languages? If you step back, it's kind of weird that there's no mainstream programming language that has tables as first class citizens. Instead, we're stuck learning multiple APIs (polars, pandas) which are effectively programming languages for tables. R is perhaps the closest, because it has data.frame as a 'firs…
There are a number of dynamic languages to choose from where tables/dataframes are truly first-class datatypes: perhaps most notably Q[0]. There are also emerging languages like Rye[1] or my own Lil[2]. I suspect that in the fullness of time, mainstream languages will eventually fully incorporate tabular programming in much the same way they have slowly absorbed a variety of idioms traditionally seen as part of funct…
Interesting links - tnx. Apropos the optimism of "eventually", I think of language support for say key-value pair collections, namespaces, as still quite impoverished. With each language supporting only a small subset of the concision, apis, and datastructures, found useful in some other. This some 3 decades after becoming mainstream, and the core of multiple mainstream languages. Diminishing returns, silos, segregation of application domains, divergence of paradigm/orientation/idioms, assorted dysfunctions as a field, etc... "eventually" can be decades. Maybe LLMs can quicken that... or perhaps call an end to this era, permitting a "no, we collectively just never got around to creating any one language which supported all of {X}".
Re: Python is not a great language for data science
#297The pure Python code in the last example is more verbose than it needs to be. groups = {} for row in filtered: key = (row['species'], row['island']) if key not in groups: groups[key] = [] groups[key].append(row['body_mass_g']) can be rewritten as: groups = collections.defaultdict(list) for row in filtered: groups[(row['species'], row['island'])].append(row['body_mass_g']) and variance = sum((x - mean) ** 2 for x in v…
There's also itertools.groupby, maybe not much shorter (need to define the keyfunc, sort, then iterate), but it does make the intent obvious.
Re: Python is not a great language for data science
#298Earlier quoted context omitted.
No, R is a serious general purpose programming language that is great for building almost any type of complex scientific software with. Projects like Bioconductor are a good example.
No, it's not. Even established packages have bugs caused by R weirdness. I like it nevertheless.
Yes, R is a proper general purpose programming language. Turing complete, functional, procedural, object oriented.../
Re: Python is not a great language for data science
#299There is a similar tread, regarding life-time of projects, such as which ecosystem is better for long-term maintainability:
https://news.ycombinator.com/item?id=46055463
Re: Python is not a great language for data science
#300Has anybody else noticed how much Python took from Scala for type hints? I was using Scala around 2015 and when I see type hints, immediately recognise its similarity to Scala's approach.