What are the implications of the new stringdtype? If I remember correctly, string performance was a big part of the pandas switch to arrow.
NumPy 2.0
61–70 of 78 posts
Re: NumPy 2.0
#62I wish numpy pushed their structured arrays (and thereby also improvements to their interface) more aggressively.
Most people are simply unaware of them, which is why we get stuff like pandas on top of everything.
Re: NumPy 2.0
#63Earlier quoted context omitted.
From a consumer (developer consumer) point of view, I hear you. From a project point of view, there are some pretty strong contra-indicators in the last 20 years of language development that make this plan suspect, or at least pretty scary — both Perl and Python had extremely rocky transitions around major versions; Perl’s ultimately failing and Python’s ultimately taking like 10 years. At least. I think the last tim…
i think it’s fair to say that perl6 has been an “extremely rocky transition” ultimately it was renamed raku to reflect this and avoid camping on the perl5 version numbering raku has good package compatibility via Inline::Perl5 and Inline::Python and FFI to languages like Rust and Zig among the many downsides of the transition, one upside is that raku is a clean sheet of paper and has some interesting new work for exa…
Thanks for this — Perl 4 was my first serious programming language, and every five or so years I come back and check out what’s up. Seems like a quick raku tour could be fun.
One huge pain point for me in perl 5 was just how incredibly slow CPAN was compared to `go import`, like two orders of magnitude slower. I remember putting up with those times in the ‘90s because package management was a kind of miracle over FTP sites, but it’s a big ask in today’s world.
What’s raku’s story here, out of curiosity?
Re: NumPy 2.0
#64Earlier quoted context omitted.
Explicit internal broadcasting (by adding an axis with an explicit `indefinite` size instead of `1`) would be so much simpler to reason about. Unfortunately there is far too much existing code and python is not type-safe.
You can do this with `np.newaxis` - in the NumPy course I wrote as TA we required students to always be explicit about the axes (also in e.g. sums). It would be nice if you could disable the implicit broadcasting, but as you mention that would break so much code
`np.newaxis` explicitly adds a `1` dimension, not an `indefinite` one.
Re: NumPy 2.0
#65Earlier quoted context omitted.
ChatGPT is really good at this. Using it to solve numpy and matplotlib problems is worth the cost of the subscription.
What a waste of electricity. All because numpy and pandas did not get their APIs right. My hot take.
I'd say you're both right, based on my limited experience. I can't seem to do much in numpy or pandas without resorting to ChatGPT or a notebook-based assistant. I assume that need will go away with additional experience, but maybe that's being overoptimistic.
I wish ChatGPT had been around when I learned C. It would sure have saved the programmers in my neighboring offices a lot of grief.
Re: NumPy 2.0
#66I would love for numpy to be ported as a typescript project personally. So I can do ml in ts. The python ecosystem feels a bit insane to me (more so than the js one). Venv helps but is still inferior to a half decent npm project imo. I feel there is no strict reason why this migration couldn't happen, only the inertia that makes it unlikely
Re: NumPy 2.0
#67it feels like the first major release in 18 years which introduces lots of breaking changes should just be a fork rather than a version. let me do `pip install numpy2` and not have to worry about whether or not some other library in my project requires numpy<2.
That was my first reaction as well, but apparently as far a Python goes, numpy 2 code is fully compatible with numpy 1 code [0], with exception of single "byte_bounds" function (which sounds super rare, so I doubt it'd be a problem)
So at least the migration path for python modules is clear: upgrade to be numpy 2 compatible, wait for critical mass, start adding numpy 2 features. Sounds way better than python2 -> python3 migration, for example.
However, the fact that I had to look at 3rd party page to find this out is IMHO a big documentation problem. It should be plastered in all announcements, on documentation and migration page: "there is a common subset for python code of numpy 1 and 2, so you can upgrade now, no need to wait for full adoption"
Re: NumPy 2.0
#68I would love for numpy to be ported as a typescript project personally. So I can do ml in ts. The python ecosystem feels a bit insane to me (more so than the js one). Venv helps but is still inferior to a half decent npm project imo. I feel there is no strict reason why this migration couldn't happen, only the inertia that makes it unlikely
who's stopping you
Re: NumPy 2.0
#69Earlier quoted context omitted.
i think it’s fair to say that perl6 has been an “extremely rocky transition” ultimately it was renamed raku to reflect this and avoid camping on the perl5 version numbering raku has good package compatibility via Inline::Perl5 and Inline::Python and FFI to languages like Rust and Zig among the many downsides of the transition, one upside is that raku is a clean sheet of paper and has some interesting new work for exa…
Thanks for this — Perl 4 was my first serious programming language, and every five or so years I come back and check out what’s up. Seems like a quick raku tour could be fun. One huge pain point for me in perl 5 was just how incredibly slow CPAN was compared to `go import`, like two orders of magnitude slower. I remember putting up with those times in the ‘90s because package management was a kind of miracle over FTP…
(I suggest you start with https://raku.guide (like the Rust Book) and also take a look at https://docs.raku.org/language/5to6-nutshell)
the raku package manager - zef comes bundled with the rakudo compiler - I use https://rakubrew.org
https://raku.land is a directory of raku packages
I would say that zef is very good (it avoids the frustrations of Python package managers like pip and conda) like perl before it, raku was designed with packages and installers in mind with a concern for a healthy ecosystem
for example, all versions (via the META6.json payload descriptor) carry versioning and the module version descriptor is a built in language type https://docs.raku.org/type/Version that does stuff like this:
say v1.0.1 ~~ v1.*.1; # OUTPUT: «True»
zef install Dan::Pandas:ver:auth:api
use Dan::Pandas:ver:auth:api;
Re: NumPy 2.0
#70The thing I want most is a more sane and more memorable way to compose non-element-wise operations. There are so many different ways to build views and multiply arrays that I can’t remember them and never know which to use, and have to relearn them every time I use numpy… broadcasting, padding, repeating, slicing, stacking, transposing, outers, inners, dots of all sorts, and half the stack overflow answers lead to th…
Most of the bugs I got on numpy programs came from a variable with a different ndims as expected being broadcast implicitly. Implicit type casting is considered a mistake in most programming languages; if I were to redesign numpy from scratch I would make all broadcasting explicit. My solution to these problems is asserting an array's shape often. Does anybody know is there's a tool like mypy or valgrind, but that ch…
numpy's API was carefully designed to match matlab, where most of its early users came from.