Scaling NumPy on Free-Threaded Python
labs.quansight.org
Scaling NumPy on Free-Threaded Python
1–10 of 27 posts
Re: Scaling NumPy on Free-Threaded Python
#2[flagged]
Re: Scaling NumPy on Free-Threaded Python
#3Nice to see the performance improvements work.
Re: Scaling NumPy on Free-Threaded Python
#4[flagged]
Perhaps, however, this hacky language is more productive in the real world than most. What’s the point, otherwise?
Re: Scaling NumPy on Free-Threaded Python
#5This is well-written. I could follow along quite nicely, from the setup through the bottlenecks and onto the resolution of the performance bug. Even the PRs are very pleasant to read: the majority of them is just a handful of changed lines with an added tests and a bit of documentation.
I was taken aback for a moment that this work originated from a report on StackOverflow. I had thought SO was effectively dead and abandoned by its community. But maybe I shouldn't project my own experience onto everyone else.
Re: Scaling NumPy on Free-Threaded Python
#6[flagged]
All of software is a hack in some way or another. You think of tradeoffs and make a decision. Theoretical purity is mostly an academic thing and Im sure you would love TCS.
Re: Scaling NumPy on Free-Threaded Python
#7[flagged]
Perhaps, however, this hacky language is more productive in the real world than most. What’s the point, otherwise?
The points:
- As the GP stated, the thrill of hidden bugs.
- Feeling productive due to fixing eternal issues.
- Getting paid to fix eternal issues.
- Feeling smart by talking about unnecessary issues.
- Writing a constant stream of PEPs to fix issues.
- Give conference talks about how you fixed issues.
- Give conference talks about how you will speed up Python by 5%.
Re: Scaling NumPy on Free-Threaded Python
#8> only acquire the lock when the flag needs to be updated
Unclear why you still need the lock here in that case. The idea that this flag may get updated during runtime and impacts how the software works when set seems to clash with the idea we need take no action having performed a relaxed (ie non-synchronising) load and seen it wasn't set at some previous time.
Maybe there's something I don't understand about these internals, which may be as simple as "It's just advisory so if we don't trace when we should no big deal".
Re: Scaling NumPy on Free-Threaded Python
#9I know this is more or less expected, but the improvement induced by adding a worker diminishes very rapidly... I guess it's not the cpython/numpy's fault but rather the CPU.
Re: Scaling NumPy on Free-Threaded Python
#10I know this is more or less expected, but the improvement induced by adding a worker diminishes very rapidly... I guess it's not the cpython/numpy's fault but rather the CPU.
The more fundamental reason is Amdahl's law
https://en.wikipedia.org/wiki/Amdahl%27s_law
Even a tiny bit of serial instruction will limit the speed up