Is there a good overview of the bigger picture here?
https://peps.python.org/pep-0703/
Related:
Intent to approve PEP 703: making the GIL optional - https://news.ycombinator.com/item?id=36913328 - July 2023 (499 comments)
gh-116167: Allow disabling the GIL
61–70 of 259 posts
Re: gh-116167: Allow disabling the GIL
#62Earlier quoted context omitted.
And I guess what I don't understand is why people choose Python for these use cases. I am not in the "Rustify" everything camp, but Go + C, Java + JNI, Rust, and C++ all seem like more suitable solutions.
Notably, all of those are static languages and none of them have array types as nice as PyTorch or NumPy, among many other packages in the Python ecosystem. Those two facts are likely closely related.
Python is just the more popular language. Julia array manipulation is mostly better (better syntax, better integration, larger standard library) or as good as python. Julia is also dynamically typed. It is also faster than Python, except for the jit issues.
Re: gh-116167: Allow disabling the GIL
#63While the title is correct, it is a bit misleading, because disabling the GIL breaks the asyncio tests. It's like saying the engine can be removed from my car. Sure, it can, but the car won't work.
Being able to remove the engine from my car with the push of a button would be a pretty amazing feature!
Analogy breaking down and all, but …
Only as long at it’s as easy to put back in
Re: gh-116167: Allow disabling the GIL
#64I've been programming in Python for over 6 years now and every week I learn something new. But recently I've been thinking about moving to a more capable language with proper concurrency for backend API requests (FastAPI sucked). I also want types, so Elixir is not in the picture. I dabbled in Rust a bit. Although I was able to get the hang of things and build a CLI tool pretty quickly, I'm worried I'll have to deal…
> Python+Types+Concurrency
Sounds like Groovy. But I wouldn't recommend it. Also the career padding hype is gone.
Re: gh-116167: Allow disabling the GIL
#65Earlier quoted context omitted.
Well this release will break any code that uses threads. The goal of this particular release is to work for thread-free programs.
This isn't correct. TFA said that small threaded programs had been run successfully, but that the test suite broke in asyncio. Async I/O and threads are two different things, and either can be present in real code without the other.
Not quite sure what your comment means exactly or how it implies what I said is incorrect.
At any rate, test_asyncio contains a lot of tests that involve threads and specifically thread safety between coroutines and those tests fail. As far as async I/O and threads being distinct, I mean sure that is true of a lot of features but people mix features together and mixing asyncio with threads will not work with this particular release.
Re: gh-116167: Allow disabling the GIL
#66Earlier quoted context omitted.
multiprocessing only works fine when you're working on problems that don't require 10+ GB of memory per process . Once you have significant memory usage, you really need to find a way to share that memory across multiple CPU cores. For non-trivial data structures partly implemented in C++ (as optimization, because pure python would be too slow), that means messing with allocators and shared memory. Such GIL-workaroun…
"Ray" can share python objects memory between processes. It's also much easier to use than multi processing.
How does that work? I'm not familiar with Ray, but I'm assuming you might be referring to actors [1]? Isn't that basically the same idea as multiprocessing's Managers [2], which also allow client processes to manipulate a remote object through message-passing? (See also DCOM.)
[1] https://docs.ray.io/en/latest/ray-core/walkthrough.html#call...
[2] https://docs.python.org/3/library/multiprocessing.html#manag...
Re: gh-116167: Allow disabling the GIL
#67Although this is nice, the problems with the GIL are often blown out of proportion: people stating that you couldn't do efficient (compute-bounded) multi-processing, which was never the case as the `multiprocessing` module works just fine.
`multiprocessing` works fine for serving HTTP requests or do some other subset of embarrassingly-parallel problems.
> `multiprocessing` works fine for serving HTTP requests
Not if you use Windows, then it's a mess. I have a suspicion that people who say that the multiprocessing works just fine never had to seriously use Python on Windows.
Re: gh-116167: Allow disabling the GIL
#68Earlier quoted context omitted.
I wish I had your optimism. Thoughtless bandwagon-y "criticism" is extraordinarily persistent.
There's no need to pretend Python has virtues which it lacks. It's not a fast language. It's fast enough for many purposes, sure, but it isn't fast, and this work is unlikely to change that. Fast er , sure, and that's great.
You seem to be implying that there is something inherently slow to Python. What?
This topic is an example: a detail of one particular implementation, since GIL is definitely not inherent to the language. Just the usual worry about looseness of types?
Re: gh-116167: Allow disabling the GIL
#69I've been programming in Python for over 6 years now and every week I learn something new. But recently I've been thinking about moving to a more capable language with proper concurrency for backend API requests (FastAPI sucked). I also want types, so Elixir is not in the picture. I dabbled in Rust a bit. Although I was able to get the hang of things and build a CLI tool pretty quickly, I'm worried I'll have to deal…
What sucked about FastAPI?