Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intent to approve PEP 703: making the GIL optional

discuss.python.org

Post reply on HN
Print view View original

191–200 of 513 posts

Re: Intent to approve PEP 703: making the GIL optional

#191

Post by pseudocomposer »

eru wrote: post #105
neilv wrote: post #61

Earlier quoted context omitted.

I remember scouring those C runtime docs, for every non-reentrant function. It might be what got me in the habit of checking docs when using some API that I know moderately well, just in case there's some important detail I missed before, or something had changed. Around that time, doing cross-platform C++, I got an early look at Java, with concurrency built in from the start, along with GC and various other nice fea…

> I got an early look at Java, with concurrency built in from the start, [...] and I "knew" it was going to be huge. And Java doesn't even have good (conceptual) support for concurrency. Compared with eg Erlang, Rust or even Haskell. But Java was still better at it than C or C++ at the time.

I agree that Rust handles concurrency about as well as Java (in addition to its memory safety). But could you provide docs on Erlang or Haskell concurrent data structures that were better than Java’s, say, 10-20 years ago? Java’s support for concurrent data structures was pretty unprecedented in its heyday.

Re: Intent to approve PEP 703: making the GIL optional

#192

Post by nologic01 »

kalb_almas wrote: post #145

Even with improved support for parallelism, what role will Python have in the future if Mojo makes good on even half of its promises?

Mojo is not Python.

The underlying pressure on the Python ecosystem is to transition to a post-Moore's law era and effectively become a HPC platform where the "same" code runs on a CPU, a GPU, multicore, clusters etc.

Python may feel the pressure more than others because of the GIL and the fact it is used in compute intensive tasks more than others.

But this major need to transition to easy and seamless HPC/heterogeneous computing is the same for all languages. The question is who will get there first.

Re: Intent to approve PEP 703: making the GIL optional

#194

Post by 8n4vidtmkvmk »

qbasic_forever wrote: post #170
wokwokwok wrote: post #161

Earlier quoted context omitted.

I’m not gonna argue that point; but it seems massively disingenuous to down vote someone who complains “but now I have to rewrite my library because some people might use it in non-GIL mode”. That’s not whining; it’s just an observation that the committee making these decisions gives zero ducks about the impact this will have for anyone other than the handful of vested parties involved in making the decisions. Pypi h…

Yep I fully agree. It's going to ultimately mean 99% of people end up running in old GIL mode with deterministic behavior. Companies will get burned and have to have policies that absolutely under no circumstances will the GIL be disabled in their codebase. A very small handful of highly skilled and funded teams, probably at big companies only, will have the time and tenacity to make their code AND all their dependen…

I work at a big company. We had probably millions of lines of Python. We migrated to c++ instead of to Python 3.

Re: Intent to approve PEP 703: making the GIL optional

#195

Post by mikepurvis »

paulddraper wrote: post #63
ggm wrote: post #33

Lots of C library code for decades carried man page warnings it was unstable for use in async, re-entrant and recursive contexts. We learned how to cope and incrementally re-entrant safe versions deployed without too much API instability. Maybe time has healed wounds and caused me memory loss of the pain of discovery you'd tripped over them. String parsing which tokenised in-place. DNS calls which used static buffers…

No-GIL mode is optional, and libraries will be marked "no-GIL compatible" and the ecosystem will gradually support more and more of these. No one's flipping a switch and breaking mountains of sketchy C.

Coming from a place of total ignorance, it would be nice if you could do this more incrementally, like have it be in no-gil mode by default, but then have a context manager you can use for gil sections, and have the interpreter bomb out if you try to enter gil-required code while still in no-gil mode.

Re: Intent to approve PEP 703: making the GIL optional

#196

Post by vlz »

slt2021 wrote: post #133
travisjungroth wrote: post #82

Earlier quoted context omitted.

I’ve seen no description of how this won’t be like 2 -> 3 except: 1. We don’t want it to be. 2. We’ll give up quickly if it is. Those are both important points. But there seems to be an important missing third piece of “and we’ll achieve this by…”.

quite a large set of users are using default python installation. There is large number of companies and non-trivial codebases that run on default python that comes with the oldest LTS version of Linux distro. currently quite a lot of companies use python3.6 only because it comes standard with the Ubuntu 14.04.6 which happens to be the oldest LTS version - and companies have habit of migrating from out-of-support LTS…

But shouldn't old GIL python versions be fine running code that is no-GIL compatible (assuming it is otherwise compatible)? Having a thread per process doesn't mean you cannot run code which is fine with having n threads per process. So if you maintain code which is otherwise compatible with e.g. python 3.6, after making it no-GIL compatible it should still be compatible to 3.6.

Re: Intent to approve PEP 703: making the GIL optional

#197

Post by pdntspa »

c-hendricks wrote: post #190
nequo wrote: post #141

Earlier quoted context omitted.

Without the last sentence, your comment would be better.

GP would probably be better if they didn't default to snark / derision by calling it cat hearding.

But the python 2->3 migration has been exactly that

Re: Intent to approve PEP 703: making the GIL optional

#198

Post by HoppyHaus »

mikepurvis wrote: post #195
paulddraper wrote: post #63

Earlier quoted context omitted.

No-GIL mode is optional, and libraries will be marked "no-GIL compatible" and the ecosystem will gradually support more and more of these. No one's flipping a switch and breaking mountains of sketchy C.

Coming from a place of total ignorance, it would be nice if you could do this more incrementally, like have it be in no-gil mode by default, but then have a context manager you can use for gil sections, and have the interpreter bomb out if you try to enter gil-required code while still in no-gil mode.

The issue is when someone's unattended-upgrades bumps up the version and causes something to come crashing down. The people who need to use nogil should know that they need to, and will now have the ability to enable it

Re: Intent to approve PEP 703: making the GIL optional

#199

Post by paulddraper »

qbasic_forever wrote: post #132
paulddraper wrote: post #127

Earlier quoted context omitted.

> how does one library support _both_ python 2 and 3 from one codebase How does one library support both GIL and no-GIL? Easy, it supports no-GIL, so it supports both. Done.

You're saying, "just rewrite and re-release all your dependencies, it's easy!" which was exactly what happened disastrously with python 2 to 3.

No, I'm saying that the problem you worry about doesn't exist.

The problem -- as you point out -- with 2 -> 3 was that supporting both versions was very difficult. Because Python 2 couldn't run Python 3 code (and vice versa).

And thus libraries existed in awkward states for years.

But GIL can run no-GIL code. Supporting both is no harder than supporting one of those options (the no-GIL one).

Post reply on HN