Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

gh-116167: Allow disabling the GIL

github.com

Post reply on HN
Print view View original

161–170 of 259 posts

Re: gh-116167: Allow disabling the GIL

#161

Post by dathery »

tiffanyh wrote: post #130

ELI5 I get in concept what the GIL is. But what's the impact of this change? Packages will now break, for the hope of better overall performance?

Previously people basically didn't bother to write multithreaded Python at all due to the GIL. Threads were primarily used when you had multiple pieces of work to do which could end up blocked on independent I/O. Which is common and useful of course, but doesn't help with the performance of CPU-bound Python code.

Even outside of high-intensity CPU work, this can be useful. A problem lately is that a lot of code is written using Python's native asyncio language features. These run single-threaded with async/await to yield execution, much like in NodeJS, and can achieve pretty good throughput even with a single thread (thousands of reqs/second).

However, a big problem is that any time you do _any_ CPU work, you block all other coroutines, which causes all kinds of obscure issues and ruins your reqs/second. For example, you might see random IO timeouts in one coroutine which are actually caused by a totally different coroutine hogging the CPU for a bit. It can be very hard to get observability into why this is happening. asyncio provides a `asyncio.to_thread()` function [1] which can help to take blocking work off the main thread, but because of the GIL it doesn't truly allow the CPU-bound to avoid interfering with other coroutines.

[1] https://docs.python.org/3/library/asyncio-task.html#asyncio....

Re: gh-116167: Allow disabling the GIL

#162

Post by colatkinson »

ptx wrote: post #73
skrause wrote: post #67

Earlier quoted context omitted.

> `multiprocessing` works fine for serving HTTP requests Not if you use Windows, then it's a mess. I have a suspicion that people who say that the multiprocessing works just fine never had to seriously use Python on Windows.

Why is it a mess? What's wrong with it on Windows?

Adding on to the other comment, multiprocessing is also kinda broken on Linux/Mac.

1. Because global objects are refcounted, CoW effectively isn't a thing on Linux. They did add a way to avoid this [0], but you have to manually call it once your main imports are done.

2. On Mac, turns out a lot of the system libs aren't actually fork-safe [1]. Since these get imported inadvertently all the time, Python on Mac actually uses `spawn` [2] -- so it's roughly as slow as on Windows.

I haven't worked in Python in a couple years, but handling concurrency while supporting the major OSes was a goddamn mess and a half.

[0]: https://docs.python.org/3.12/library/gc.html#gc.freeze

[1]: https://bugs.python.org/issue33725

[2]: https://docs.python.org/3.12/library/multiprocessing.html#co...

Re: gh-116167: Allow disabling the GIL

#163

Post by fragmede »

KMnO4 wrote: post #123
fragmede wrote: post #115

Earlier quoted context omitted.

tranched bread?

I could be wrong, but I think it’s a clever alternative to the expression “best thing since sliced bread”.

ahahah. I was thinking that it was a new python library or something that I hadn't heard of and was coming up short with Google.

"tranced bread" is a fun name for some sort of library that breaks up files into pieces for better resilience for sending, like over BitTorrent.

Re: gh-116167: Allow disabling the GIL

#164

Post by KaiserPro »

rmbyrro wrote: post #72
samatman wrote: post #44

Earlier quoted context omitted.

There's no need to pretend Python has virtues which it lacks. It's not a fast language. It's fast enough for many purposes, sure, but it isn't fast, and this work is unlikely to change that. Fast er , sure, and that's great.

Although true, it doesn't mean they can't improve its performance. Working with threads is a pain in Python. If you want to spawn +10-20 threads in a process, it can quickly become way slower than running a single thread. Removing the GIL and refactoring some of the core will unlock levels of concurrency that are currently not feasible with Python. And that's a great deal, in my opinion. Well worth the trouble they'r…

> If you want to spawn +10-20 threads in a process, it can quickly become way slower than running a single thread.

as you know thats mostly threads in general. Any optimisation has a drawback so you need to choose wisely.

I once made a horror of a thing that synced S3 with another S3, but not quite object store. I needed to move millions of files, but on the S3 like store every metadata operation took 3 seconds.

So I started with async (pro tip: its never a good idea to use async. its basically gotos with two dimensions of surprise: 1 when the function returns, 2 when you get an exception ) I then moved to threads, which got a tiny bit extra performance, but much easier debugability. Then I moved to multiprocess pools of threads (fuck yeah super fast) but then I started hitting network IO limits.

So then I busted out to airflow like system with operators spawning 10 processes with 500 threads.

it wasnt very memory efficient, but it moved many thousands of files a second.

Re: gh-116167: Allow disabling the GIL

#165

Post by KaiserPro »

ebiester wrote: post #35
ynik wrote: post #28

Earlier quoted context omitted.

multiprocessing only works fine when you're working on problems that don't require 10+ GB of memory per process . Once you have significant memory usage, you really need to find a way to share that memory across multiple CPU cores. For non-trivial data structures partly implemented in C++ (as optimization, because pure python would be too slow), that means messing with allocators and shared memory. Such GIL-workaroun…

And I guess what I don't understand is why people choose Python for these use cases. I am not in the "Rustify" everything camp, but Go + C, Java + JNI, Rust, and C++ all seem like more suitable solutions.

> but Go + C, Java + JNI, Rust, and C++ all seem like more suitable solutions.

apart from go (maybe java) those are all "scary" languages that require a bunch of engineering to get to the point that you can prototype.

even then you can normally pybind the bits that are compute bound.

If Microsoft had been better back in the say, then c# should have been the goto language of choice. It has the best tradeoff of speed/handholding/rapid prototyping. Its also statically typed, unless you tell it to not be.

Re: gh-116167: Allow disabling the GIL

#166

Post by aunderscored »

nextlevelwizard wrote: post #152
ralphist wrote: post #85

Earlier quoted context omitted.

It's not a separate language, you can just start typing your programs right now.

except nothing enforces your types at run time, you can have typi hints all you want and everyone else cn ignore them

Same exists with laws. And in most languages if I want to avoid the type system and hand you a goldfish instead of an int, I can. It just may take more effort. Other language will blow up too if you hand them strange things. Just like python those languages have ways to verify you're not passing goldfish, You just may need more or less effort to use them

Re: gh-116167: Allow disabling the GIL

#167

Post by KaiserPro »

behnamoh wrote: post #76
willcipriano wrote: post #69

Earlier quoted context omitted.

What sucked about FastAPI?

Not fast enough! I used it to call llama.cpp server but it would crash if requests were "too fast". Calling the llama.cpp server directly solved the issue.

Interesting, I've used fastAPI to serve many thousands of requests a second (per process) for a production system. How were you buffering the requests?

Re: gh-116167: Allow disabling the GIL

#168

Post by scubbo »

bmitc wrote: post #127
scubbo wrote: post #37

Earlier quoted context omitted.

I wish I had your optimism. Thoughtless bandwagon-y "criticism" is extraordinarily persistent.

It isn't thoughtless. I'm working in Python after having come from more designed languages, and concurrency in Python is an absolute nightmare. It feels like using a language from the 60s. An effectively single threaded language in 2024! That's really astonishing.

If your criticism isn't thoughtless, then that's not what I'm complaining about. Specifically, I'm annoyed about people who _just_ say "Python isn't fast enough, therefore it's not suitable to our use-case", when their use-case doesn't require significant speed or concurrency. If you thoughtfully discount Python as being unsuitable for a use-case that it's _actually_ unsuitable for, then good luck to you!

Re: gh-116167: Allow disabling the GIL

#169

Post by its-summertime »

nerpderp82 wrote: post #97
Hugsun wrote: post #26

It will be very exciting to see how much faster they'll be able to make vanilla python. The value proposition is being challenged by the plethora of tools aiming to alleviate those issues. Speed improvers like Mojo, pytorch, triton, numba, taichi come to mind. There are so many different attempts at solving this problem that that the last time I wanted to try one of them, I found myself overwhelmed with options. I ch…

Mojo should be viewed as an attack on the Python ecosystem due to it being a superset. It can consume Python, but it itself is not Python. Taichi is really underrated, it works across all platforms (including Metal), has tons of examples and the code is easy to write. And lastly, it integrates with the ecosystem and doesn't displace it. https://github.com/taichi-dev great demo reel of what Taichi can do, https://www.…

Cython is also a superset, is Cython also guilty of such crimes?

Re: gh-116167: Allow disabling the GIL

#170

Post by scubbo »

nick238 wrote: post #159
cmeacham98 wrote: post #48

Earlier quoted context omitted.

This seems a bit like saying "JavaScript supports types!" because of typescript.

Python actually has type safety though, as you can't do `'1' + 1` like in JS (not that a linter wouldn't scream at you). If I hear another "I compile so I know it will work, but you can't do that in Python" I'll lose it. Having the compiler not complain that the types match is not effing "testing".

> Having the compiler not complain that the types match is not effing "testing".

It absolutely is - it's just testing at a _very_ low level of correctness, and is not sufficient for testing actual high-level functionality.

Post reply on HN