Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling NumPy on Free-Threaded Python

labs.quansight.org

Post reply on HN
Print view View original

21–27 of 27 posts

Re: Scaling NumPy on Free-Threaded Python

#21

Post by ngoldbaum »

quietbritishjim wrote: post #18
frollogaston wrote: post #14

I thought NumPy was already releasing the GIL. On regular non-free-threaded Python, you can run threaded parallel Numpy operations and have multiple cores doing 100%, I've relied on that. Maybe not the case with the operations this article focuses on (sin/cos).

Yes, numpy does release the GIL. But the code in question has multiple numpy calls, called in a loop: sum((np.sin(np.cos(np.sin(np.cos(x + i)))).sum() for i in range(n_loop))) This is not like: release GIL # i = 0 compute y = x + 0 (numpy broadcasting sum) compute z = np.cos(y) (elementwise) ... add to running total # i = 1 compute y = x + 1 ... reacquire GIL Instead it is: # i = 0 look up "+" operation release GIL c…

There’s also the fact that INCREF and DECREF on shared objects is a lot more expensive than plain integer addition, so stuff becomes a bottleneck that was never a bottleneck. Kumar also fixed a bottleneck caused by a lock added only for safety on the free-threaded build. It’s hard to tell in advance than a fancy lock-free data structure is needed for something.

Re: Scaling NumPy on Free-Threaded Python

#22

Post by frollogaston »

quietbritishjim wrote: post #18
frollogaston wrote: post #14

I thought NumPy was already releasing the GIL. On regular non-free-threaded Python, you can run threaded parallel Numpy operations and have multiple cores doing 100%, I've relied on that. Maybe not the case with the operations this article focuses on (sin/cos).

Yes, numpy does release the GIL. But the code in question has multiple numpy calls, called in a loop: sum((np.sin(np.cos(np.sin(np.cos(x + i)))).sum() for i in range(n_loop))) This is not like: release GIL # i = 0 compute y = x + 0 (numpy broadcasting sum) compute z = np.cos(y) (elementwise) ... add to running total # i = 1 compute y = x + 1 ... reacquire GIL Instead it is: # i = 0 look up "+" operation release GIL c…

Ah ok, the times I did this and saw full CPU utilization was on pretty large arrays, so there was a lot less time spent in the GIL.

Re: Scaling NumPy on Free-Threaded Python

#23

Post by Grimblewald »

inigyou wrote: post #20
w-m wrote: post #5

This is well-written. I could follow along quite nicely, from the setup through the bottlenecks and onto the resolution of the performance bug. Even the PRs are very pleasant to read: the majority of them is just a handful of changed lines with an added tests and a bit of documentation. I was taken aback for a moment that this work originated from a report on StackOverflow. I had thought SO was effectively dead and a…

SO is dead and abandoned by its community, and the data proves it. https://data.stackexchange.com/stackoverflow/query/1882532/q...

or maybe SO is back to its community sans clout chasers and tourists? If a vaninishing minority drive a given community in content generation and discourse, then lurkers etc leaving isnt as meaningful. you could lose 99% of users on many platforms without disrupting the core community and often having the added benefit of imoroving SNR.

It's a less marketable product in the modern attention economy, but that isnt the same as a dead community. It would be interesting to see a plot of posts/discussion liveliness per unit time and seeing is quality and depth of both questions and answers has changed and how they have changed.

Re: Scaling NumPy on Free-Threaded Python

#24

Post by inigyou »

Grimblewald wrote: post #23
inigyou wrote: post #20

Earlier quoted context omitted.

SO is dead and abandoned by its community, and the data proves it. https://data.stackexchange.com/stackoverflow/query/1882532/q...

or maybe SO is back to its community sans clout chasers and tourists? If a vaninishing minority drive a given community in content generation and discourse, then lurkers etc leaving isnt as meaningful. you could lose 99% of users on many platforms without disrupting the core community and often having the added benefit of imoroving SNR. It's a less marketable product in the modern attention economy, but that isnt the…

this is copium

Re: Scaling NumPy on Free-Threaded Python

#25

Post by quietbritishjim »

ngoldbaum wrote: post #21
quietbritishjim wrote: post #18

Earlier quoted context omitted.

Yes, numpy does release the GIL. But the code in question has multiple numpy calls, called in a loop: sum((np.sin(np.cos(np.sin(np.cos(x + i)))).sum() for i in range(n_loop))) This is not like: release GIL # i = 0 compute y = x + 0 (numpy broadcasting sum) compute z = np.cos(y) (elementwise) ... add to running total # i = 1 compute y = x + 1 ... reacquire GIL Instead it is: # i = 0 look up "+" operation release GIL c…

There’s also the fact that INCREF and DECREF on shared objects is a lot more expensive than plain integer addition, so stuff becomes a bottleneck that was never a bottleneck. Kumar also fixed a bottleneck caused by a lock added only for safety on the free-threaded build. It’s hard to tell in advance than a fancy lock-free data structure is needed for something.

You're talking about the same thing as the article: why were things not as fast as they could be in free threaded mode?

But this comment thread is more of a meta discussion: why would lock contention issues only show up now, in free threaded mode, if numpy already released the GIL anyway? That's what I answered above: yes the GIL was sometimes released by numpy, but these operations really were happening with the GIL locked (in non free threaded build).

Re: Scaling NumPy on Free-Threaded Python

#26

Post by ngoldbaum »

quietbritishjim wrote: post #25
ngoldbaum wrote: post #21

Earlier quoted context omitted.

There’s also the fact that INCREF and DECREF on shared objects is a lot more expensive than plain integer addition, so stuff becomes a bottleneck that was never a bottleneck. Kumar also fixed a bottleneck caused by a lock added only for safety on the free-threaded build. It’s hard to tell in advance than a fancy lock-free data structure is needed for something.

You're talking about the same thing as the article: why were things not as fast as they could be in free threaded mode? But this comment thread is more of a meta discussion: why would lock contention issues only show up now, in free threaded mode, if numpy already released the GIL anyway? That's what I answered above: yes the GIL was sometimes released by numpy, but these operations really were happening with the GIL…

I’m saying that it’s a new issue on the free-threaded build.

Re: Scaling NumPy on Free-Threaded Python

#27

Post by quietbritishjim »

ngoldbaum wrote: post #26
quietbritishjim wrote: post #25

Earlier quoted context omitted.

You're talking about the same thing as the article: why were things not as fast as they could be in free threaded mode? But this comment thread is more of a meta discussion: why would lock contention issues only show up now, in free threaded mode, if numpy already released the GIL anyway? That's what I answered above: yes the GIL was sometimes released by numpy, but these operations really were happening with the GIL…

I’m saying that it’s a new issue on the free-threaded build.

Right. The original commenter already seemed to understand that.

It's a new issue because the reference count needs locking, whereas previously it didn't because it was implicitly synchronised due to the GIL being locked.

But the original commenter asked: hang on, I thought the GIL wasn't locked for numpy?

Now you have reached the start of the conversation.

Post reply on HN