Intent to approve PEP 703: making the GIL optional • Hacker News Board

Re: Intent to approve PEP 703: making the GIL optional

#511

Post by kerkeslager » Tue, Aug 01, 2023, 3:04 PM UTC

kerkeslager wrote: post #510

commonlisp94 wrote: post #509

Earlier quoted context omitted.

It's your own test, I don't need see why you need to respond in this manner. > Running 3000 processes is a few orders of magnitude less than running 1000000 I didn't decide on the limit, my OS did. So they must think it's unreasonable. > Running on MacOS, but I've run this in Linux. Well your test works fine on linux. MacOS is not designed to run large multi-process server loads. Linux has specifically optimized fork…

> I didn't decide on the limit, my OS did. So they must think it's unreasonable. > Well your test works fine on linux. MacOS is not designed to run large multi-process server loads. Linux has specifically optimized forking and context switching for processes. Oh JFC, stop wildly speculating and pretending it's the truth. The test was a million threads/processes, and you ran 3000. By your own description you didn't ru…

I'm just now noticing this, from your test:

   $ gcc threads.c 
   $ time ./a.out 

   real 0m10.097s
   user 0m0.035s
   sys 0m0.239s

   $ gcc process.c 
   $ time ./a.out

   real 0m10.168s
   user 0m0.579s
   sys  0m0.347s

You were pleasantly surprised by your own test showing an order of magnitude speed gain in user space by using threads?

Re: Intent to approve PEP 703: making the GIL optional

#512

Post by KolenCh » Tue, Aug 01, 2023, 11:30 PM UTC

nirvdrum wrote: post #432

bigbillheck wrote: post #425

Earlier quoted context omitted.

I don't think it's reasonable to expect people here to cater to a least common denominator of comprehension.

That's fine. But, communication is a skill and like all skills, can be improved. For my part, this is the first time I can remember seeing MIS used in the wild (i.e., not in university). Given the tech industry's propensity for overloading initialisms and redefining terms, I have to go by context to work out the meaning. In this case, the context was insufficient. I suppose I could just dismiss the comment, but curio…

Speaking of which I never fully understand the meaning of OP. It seems there’s a few semantic ranges there and people sometimes use OP standing for slightly different things.

Re: Intent to approve PEP 703: making the GIL optional

#513

Post by collinmanderson » Fri, Aug 04, 2023, 7:23 PM UTC

paulddraper wrote: post #63

ggm wrote: post #33

Lots of C library code for decades carried man page warnings it was unstable for use in async, re-entrant and recursive contexts. We learned how to cope and incrementally re-entrant safe versions deployed without too much API instability. Maybe time has healed wounds and caused me memory loss of the pain of discovery you'd tripped over them. String parsing which tokenised in-place. DNS calls which used static buffers…

No-GIL mode is optional, and libraries will be marked "no-GIL compatible" and the ecosystem will gradually support more and more of these. No one's flipping a switch and breaking mountains of sketchy C.

> No-GIL mode is optional

Long term the plan is to 100% remove the GIL from python.

From the article:

> Long-term, we want no-GIL to be the default, and to remove any vestiges of the GIL (without unnecessarily breaking backward compatibility). We don’t want to wait too long with this, because having two common build modes may be a heavy burden on the community (as, for example, it can double test resources and debugging scenarios), but we can’t rush it either. We think it may take as much as five years to get to this stage.

Under base assumptions it also says:

> We want to be very careful with backward compatibility. We do not want another Python 3 situation, so any changes in third-party code needed to accommodate no-GIL builds should just work in with-GIL builds (although backward compatibility with older Python versions will still need to be addressed).