Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intent to approve PEP 703: making the GIL optional

discuss.python.org

Post reply on HN
Print view View original

391–400 of 513 posts

Re: Intent to approve PEP 703: making the GIL optional

#391

Post by samus »

miraculixx wrote: post #346
r0l1 wrote: post #325

Earlier quoted context omitted.

https://peps.python.org/pep-0703/ Quote: "In PyTorch, Python is commonly used to orchestrate ~8 GPUs and ~64 CPU threads, growing to 4k GPUs and 32k CPU threads for big models. While the heavy lifting is done outside of Python, the speed of GPUs makes even just the orchestration in Python not scalable. We often end up with 72 processes in place of one because of the GIL. Logging, debugging, and performance tuning are…

This requirement could have been well served with a gil per thread and arena based (shared) object allocation model. Every other use case would have been unaffected. Now we change the world for everyone and put most of library developers through a valley of desperation for 5 years+, just so that a very few narrow use cases get the benefits they want. Not a smart move IMHO.

The use cases of the ML and AI world are very important though, as they massively contribute to Python's popularity. Thanks to Python, researchers and developers don't have to use different languages and library ecosystems for developing and scaling models.

Alas, subinterpreters sound like they could be a feasible solution for many use cases as well.

Re: Intent to approve PEP 703: making the GIL optional

#392

Post by mananaysiempre »

xmcqdpt2 wrote: post #368
sverhagen wrote: post #351

Earlier quoted context omitted.

While Python has never been my primary coding language, I've used it extensively for building scripts and tools, but I've pretty much given up on it. The language is so elegant, but the installation of it (with 2~3 compatibility issues being just a small part of that) just became such a turnoff. It's been super frustrating having to search the Internet every time I need to install something Python-related, to then fi…

The new official Scala build tool / compiler front end (scala-cli) is amazing, https://scala-cli.virtuslab.org/ The thing that really struck me after years of python is how it lets you out dependencies directly in a comment on top of a script and it will download and run with them automatically, without poisoning any system settings. It's so simple!

> it lets you out dependencies directly in a comment on top of a script and it will download and run with them automatically, without poisoning any system settings

The old-style nix-shell command in Nix can do this[1] for every language and package Nixpkgs supports, although it’s not that often used because it ties your shebangs to Nix. (An equivalent feature for the new CLI is a work in progress[2].)

[1] https://nixos.org/manual/nix/stable/command-ref/nix-shell.ht...

[2] https://github.com/NixOS/nix/pull/5189

Re: Intent to approve PEP 703: making the GIL optional

#393

Post by xmcqdpt2 »

slt2021 wrote: post #78
miraculixx wrote: post #74

Earlier quoted context omitted.

Fully agree. People put far too much emphasis & expectations on the "free" part in "free multithreading".

even if they get "free multithreading" with no-GIL, their system eventually will overgrow one beefy machine and will need to be deployed across a fleet of 10/100/1000 machines. at which point you lose benefit of no-GIL, because you now have to introduce redis and kafka into the system

Yes well if you are distributing to N machines, you probably want to use all M cores on each of those machines. You'll still get a performance advantage from multi-threading.

You might think that you can simply spin M processes per machine instead but now you have N*M servers instead of N servers that are M times faster. In many cases this means you have significantly higher overheads: slower startups, a lot more RAM usage, more network IO etc.

Outside of a few embarrassingly parallel problem, two-level parallelism is usually the highest performance approach.

Re: Intent to approve PEP 703: making the GIL optional

#394

Post by DonHopkins »

concordDance wrote: post #296
JonChesterfield wrote: post #242

Earlier quoted context omitted.

I hope it'll be lua. I fear it'll be javascript.

Lua? Really? That's by far the most unpleasant language I've coded in.

Obviously there are a LOT of languages you've never coded in.

Re: Intent to approve PEP 703: making the GIL optional

#395

Post by mkoubaa »

bjourne wrote: post #96

This can (and I think will) cause issues for C extensions because many are written without multi-threading in mind. Here is a small example which is unsafe if lst can be accessed from another thread: https://news.ycombinator.com/item?id=36649769 Note that the code may cause a context switch even today if the C code callbacks into Python bytecode (via a __del__ method) and the bytecode is long enough (100 instructions…

Might be a good time to write concurrency first C extension to compete with the legacy ones

Re: Intent to approve PEP 703: making the GIL optional

#396

Post by samus »

mappu wrote: post #13

With PEP703 you would compile Python either for multi or single-threading mode. The mode affects the ABI and therefore which C extensions are available. Eventually all C extensions would have an available port to the new ABI. The chosen solution is similar to how PHP used TSRMLS_ macros in the Zend engine - if threadsafety (ZTS) was #defined, all functions took an extra thread context parameter, breaking ABI.

All C extensions are available when running without GIL. A challenge for distribution is that all extensions will have to built twice to be compatible with the two Python builds. However, few source code changes are required where the developers don't want to make it compatible with running without GIL. Executing such extensions forces using the GIL for the whole interpreter, which is slower than the GIL-only build.

Re: Intent to approve PEP 703: making the GIL optional

#397

Post by xmcqdpt2 »

_flux wrote: post #363
slt2021 wrote: post #54

Naiive question: Who needs No-GIL when we have asyncio and multiprocessing packages ? never ever had a problem with GIL in python, always found a workaround just by spinning up ThreadPool or ProcessPool, and used async libraries when needed. is there any use case of No-GIL which is not solved by multiprocessing ? I thought Single threaded execution without overhead for concurrency primitives is the best way to high p…

I haven't use Python multiprocessing packages, so I need to ask, how does one do flexible work queues with them? I mean a situation like where in threaded context there would be code like: def determine_quest_latency(quest_name: str) -> int: return other_thread.wait_sync_job(lambda context: context.ping_quest(quest_name)) ..without needing to provide a protocol that covers each possible scenario the client might wish…

I believe you just pass objects instead, like you would in OOP, and take the hit of pickling and unpickling them every time.

If you really want to pass lambdas, you can use a third party library to pickle them

https://github.com/cloudpipe/cloudpickle

Yes, this is not great.

Re: Intent to approve PEP 703: making the GIL optional

#398

Post by xmcqdpt2 »

bjourne wrote: post #96

This can (and I think will) cause issues for C extensions because many are written without multi-threading in mind. Here is a small example which is unsafe if lst can be accessed from another thread: https://news.ycombinator.com/item?id=36649769 Note that the code may cause a context switch even today if the C code callbacks into Python bytecode (via a __del__ method) and the bytecode is long enough (100 instructions…

> Note that the code may cause a context switch even today if the C code callbacks into Python bytecode (via a __del__ method) and the bytecode is long enough (100 instructions I think). However, that is extremely unlikely and much C extension code is not written with such situations in mind.

As someone who works professionally on a parallel / async runtime that supports thousands of continuously running servers, "extremely unlikely" means that actually it breaks all the time, but it's also impossible to debug.

Re: Intent to approve PEP 703: making the GIL optional

#399

Post by oars »

slt2021 wrote: post #54

Naiive question: Who needs No-GIL when we have asyncio and multiprocessing packages ? never ever had a problem with GIL in python, always found a workaround just by spinning up ThreadPool or ProcessPool, and used async libraries when needed. is there any use case of No-GIL which is not solved by multiprocessing ? I thought Single threaded execution without overhead for concurrency primitives is the best way to high p…

From PEP 703:

> Manuel Kroiss, software engineer at DeepMind on the reinforcement learning team, describes how the bottlenecks posed by the GIL lead to rewriting Python codebases in C++, making the code less accessible:

> "We frequently battle issues with the Python GIL at DeepMind. In many of our applications, we would like to run on the order of 50-100 threads per process. However, we often see that even with fewer than 10 threads the GIL becomes the bottleneck. To work around this problem, we sometimes use subprocesses, but in many cases the inter-process communication becomes too big of an overhead. To deal with the GIL, we usually end up translating large parts of our Python codebase into C++. This is undesirable because it makes the code less accessible to researchers."

For average usage like web apps, no-GIL can be solved by multiprocessing. But for AI workloads at huge scale like Google and DeepMind, the GIL really does limit their usage of Python (hence the need to translate to C++). This is also why Meta are willing to commit three engineer-years to making this happen: https://news.ycombinator.com/item?id=36643670

Re: Intent to approve PEP 703: making the GIL optional

#400

Post by the8472 »

slt2021 wrote: post #54

Naiive question: Who needs No-GIL when we have asyncio and multiprocessing packages ? never ever had a problem with GIL in python, always found a workaround just by spinning up ThreadPool or ProcessPool, and used async libraries when needed. is there any use case of No-GIL which is not solved by multiprocessing ? I thought Single threaded execution without overhead for concurrency primitives is the best way to high p…

> I thought Single threaded execution without overhead for concurrency primitives is the best way to high performance computing

You can have shared-memory parallelism with near-zero synchronization overhead. Rust's rayon is an example. Take a big vector, chunk it into a few blocks, distribute the blocks across threads, let them work on it and then merge the results. Since the chunks are independent you don't need to lock accesses. The only cost you're paying is sending tasks across work queues. But that's still much cheaper than spawning a new process.

Post reply on HN