251–260 of 513 posts
Naiive question: Who needs No-GIL when we have asyncio and multiprocessing packages ? never ever had a problem with GIL in python, always found a workaround just by spinning up ThreadPool or ProcessPool, and used async libraries when needed. is there any use case of No-GIL which is not solved by multiprocessing ? I thought Single threaded execution without overhead for concurrency primitives is the best way to high p…
Earlier quoted context omitted.
no-GIL libs will work with the GIL on surely? So you won’t have to support both?
Where will you get no-GIL libraries, especially in the early days? Just yell at the maintainers of core libs like flask and requests until they use their volunteer and spare time to implement incredibly complex and tricky locking semantics all over their codebases, AND test it all with both GIL and non-GIL interpretors at scale to suss out race conditions? That just happens for free and overnight because a lot of peo…
Earlier quoted context omitted.
It's only about performance. asyncio is still inherently single-threaded, and hence also single core. multiprocessing is multi-core and hence better for performance, but each process is relatively heavy and there's additional overhead to shared memory. GIL multi-threading is both single-core and difficult to use correctly. No-GIL multi-threading is multi-core, though difficult to use. I don't know the Python implemen…
Even with multithreading Python will still be slow. It's not like singlethreaded Python can keep up with singlethreaded C or even JS. This will just allow people to waste even more compute resources to get somewhat quick results from Python instead of doing the right thing. From an ecological viewpoint this PEP will be disastrous and keep entire power plants online.
I know they say specifically that they don't want a repeat of the Python3 transition scenario, but the approach they're taking now still veers eerily close to that path, at least it looks that way to me. A lot will depend on the Python community and the distribution channels. I could see the community struggling to adopt it in a timely fashion, or distributions jumping the gun (Ubuntu, Fedora, Anaconda). Maybe it's t…
They kind of burned a breaking major version transition for no good reason with 2-to-3, now they are prefacing a major change with "it won't be like 2-to-3". It sounds like they may be maintaining two operating modes in CPython 3 instead of going forward with another major transition, just because of that history.
The unicode/text changes alone were a pretty good reason. Division producing floats are also a nice change IMO. I don’t want to discount the challenges with the transition but saying there was no good reason isn’t right to me.
Earlier quoted context omitted.
You're saying, "just rewrite and re-release all your dependencies, it's easy!" which was exactly what happened disastrously with python 2 to 3.
You can down vote this til the cows come home but it’s historically what happened. This “that’s not how it went” stuff going down is quite just blatant historical revisionism. Maybe it’s a good idea? Maybe it’s not? …but anyone down voting “this reminds me of the Python 2/3” fiasco has no idea what they’re talking about.
Historically, Napoleon lost. That has zero impact on whether I get a promotion.
The 2 changes simply have nothing to do with each other. No-GIL python can still run GIL code, it just won't be able to run it in parallel.
Earlier quoted context omitted.
> I got an early look at Java, with concurrency built in from the start, [...] and I "knew" it was going to be huge. And Java doesn't even have good (conceptual) support for concurrency. Compared with eg Erlang, Rust or even Haskell. But Java was still better at it than C or C++ at the time.
I agree that Rust handles concurrency about as well as Java (in addition to its memory safety). But could you provide docs on Erlang or Haskell concurrent data structures that were better than Java’s, say, 10-20 years ago? Java’s support for concurrent data structures was pretty unprecedented in its heyday.
Why would you even want a no-GIL Python? Java and C showed how much more effort it takes to maintain slower thread safe code for no real benefit. Parallelize at the fork level or at the isolated numeric library level.
Exactly. I think a lot of the negativity about GIL comes from a misunderstanding about forking processes. If python is being used as a scripting language, and spawning other tools, you're already getting free multi-core. A similar misunderstanding exists about SQLite and concurrency.. but that's a topic for another time.
Earlier quoted context omitted.
Where can I read up on this planned way of working? This is not what the PEP says
It's in the "Py_mod_gil Slot" section [0] in the PEP: > In --disable-gil builds, when loading an extension, CPython will check for a new PEP 489-style Py_mod_gil slot. If the slot is set to Py_mod_gil_not_used, then extension loading proceeds as normal. If the slot is not set, the interpreter pauses all threads and enables the GIL before continuing. Additionally, the interpreter will issue a visible warning naming th…
Earlier quoted context omitted.
Except they're not at all analogous situations. In python 2 -> 3, if a new version of a library came out Python 3 only, you couldn't use both it, and your old Python 2 code at the same time. Here, you can use the GIL until every one of your dependencies have migrated, even as new versions of your dependencies come out with nogil support until one magical day they all have nogil and you can migrate. But since you have…
* removed comment as it was factually false *
Testing the difference between running it with and without `nogil` without having to install 2 different interpreters.
Testing libraries during transitions.
Simply giving users a choice.
Convenience.
Almost no one uses pre-Go.1.11 (GOPATH instead of Modules) any more for project organisation, and the transition is trivially easy. And yet, the toolchain still reacts to `GO111MODULE=off`.
Earlier quoted context omitted.
no-GIL libs will work with the GIL on surely? So you won’t have to support both?
Where will you get no-GIL libraries, especially in the early days? Just yell at the maintainers of core libs like flask and requests until they use their volunteer and spare time to implement incredibly complex and tricky locking semantics all over their codebases, AND test it all with both GIL and non-GIL interpretors at scale to suss out race conditions? That just happens for free and overnight because a lot of peo…