What's up, Python? The GIL removed, a new compiler, optparse deprecated • Hacker News Board

Re: What's up, Python? The GIL removed, a new compiler, optparse deprecated

#191

Post by TylerE » Mon, Jul 31, 2023, 8:00 AM UTC

threatripper wrote: post #35

wmwmwm wrote: post #14

Historically I’ve written several services that load up some big datastructure (10s or 100s of GB), then expose an HTTP API on top of it. Every time I’ve done a quick implementation in Python of a service that then became popular (within a firm, so 100s or 1000s of clients) I’ve often ended up having to rewrite in Java so I can throw more threads at servicing the requests (often CPU heavy). I may have missed somethin…

You have a single big data structure that can't be shared easily between multiple processes. Can't you use multiprocessing with that? Maybe mapping the data structure to a file and mmapping that in multiple processes? Maybe wrapping the whole thing in database instead of just using one huge nested dictionary? To me multi-threading sounds so much less painful than all the alternatives that I could imagine. Just adding…

I'd go for a db, yeah, or if that's a really painful mapping, this, erm, is actually the sort of thing Go is pretty good at it, and it's not too hard to write a fairly simple program that will traverse your data structure and communicate via a JSON api or something. That's a useful technique in general - separate the big heavy awkward thing from your main web processes.

While I hate how verbose and inexpressive it is, Go does hit a sweet spot of fairly good performance, even multi-core, while still being GCed so it's not nearly as foreign for a native python user.

Re: What's up, Python? The GIL removed, a new compiler, optparse deprecated

#192

Post by mvncleaninst » Mon, Jul 31, 2023, 8:09 AM UTC

chlorion wrote: post #119

alfalfasprout wrote: post #93

Earlier quoted context omitted.

Nowadays multiprocessing is rarely the answer. Between all the gotchas (memory usage can be horrific, have to be careful what you modify, etc.) it's almost never the right answer. Nowadays numba is usually a better solution for when you want to run some computationally expensive python code that itself calls numpy, etc. For the parent commenter's use case though that wouldn't be a great solution either. In general, P…

You have to be much more careful about what you modify when using multithreading, so I'm not sure what you mean by that. A lot of people here mention that sharing data is much easier with multithreading, but doing this without races is not easy. You can't just use the values from difference threads like you would in normal code, you need to synchronize access with locks, which can be difficult to do correctly and can…

> Problems that can be easily parallelized already work fine with multiprocessing!

Yeah, except afaik you pay more in context switches, sharing is more cumbersome. Also language runtime of a single process is likely working with less information, you end up using more memory on multiple language runtime instances

Frankly I'd just use Java or Go at that point and not even bother

Re: What's up, Python? The GIL removed, a new compiler, optparse deprecated

#193

Post by apelapan » Mon, Jul 31, 2023, 8:11 AM UTC

dekhn wrote: post #61

nwallin wrote: post #53

Earlier quoted context omitted.

> I may have missed something but I couldn’t figure out how to get the multi-threaded performance out of Python Multiprocessing. The answer is to use the python multiprocessing module, or to spin up multiple processes behind wsgi or whatever. > Historically I’ve written several services that load up some big datastructure (10s or 100s of GB), then expose an HTTP API on top of it. Use the python multiprocessing module…

Over quite some time I've become convinced multiprocessing module is better than an optional GIL removal. It may leave many useful bits on the table (compared to pure multithreaded coding, like C++/pthreads) but I've still been able to get it to scale my application performance (CPU-bound, large-memory) to the number of cores of even large boxes (96+ vCPUs). IIRC the future/concurrent library was key to being product…

Sure, with multiprocessing you can get 96 python processes running at 100% CPU while sharing a large dataset.

Only problem is that 99% of that CPU usage is for serializing/deserializing IPC messages and total throughput would have been higher using a single process.

There are use-cases for multiprocessing. As long as data sharing between processes is insignificant, it can be quite performant. Just like using a bash-wrapper script that orchestrates a bunch of python (or other) processes.

Re: What's up, Python? The GIL removed, a new compiler, optparse deprecated

#194

Post by mort96 » Mon, Jul 31, 2023, 8:15 AM UTC

nwallin wrote: post #53

wmwmwm wrote: post #14

Historically I’ve written several services that load up some big datastructure (10s or 100s of GB), then expose an HTTP API on top of it. Every time I’ve done a quick implementation in Python of a service that then became popular (within a firm, so 100s or 1000s of clients) I’ve often ended up having to rewrite in Java so I can throw more threads at servicing the requests (often CPU heavy). I may have missed somethin…

> I may have missed something but I couldn’t figure out how to get the multi-threaded performance out of Python Multiprocessing. The answer is to use the python multiprocessing module, or to spin up multiple processes behind wsgi or whatever. > Historically I’ve written several services that load up some big datastructure (10s or 100s of GB), then expose an HTTP API on top of it. Use the python multiprocessing module…

I want to warn people against multiprocessing in python though.

If you're thinking about parallelizing your Python process, chances are your Python code is CPU-bound. That's when you should stop and think, is Python really the right tool for this job?

From experience, translating a Python program into C++ or Rust often gives a speed-up of around 100x, without introducing threads. Go probably has a similar level of speed-up. So while you can throw a lot of time fighting Python to get it to consume 16x the compute resources for a 10x speed-up, you could often instead spend a similar amount of time rewriting the program for a 100x speed-up with the same compute resources. And then you could parallelize your Go/Rust/C++ program for another 10x, if necessary.

Of course, this is highly dependent on what you're actually doing. Maybe your Python code isn't the bottleneck, maybe your code spends 99% of its time in datastructure operations implemented in C and you need to parallelize it. Or maybe your use-case is one where you could use pypy and get the required speed-up. I just recognize from my own experience the temptation of parallelizing some Python code because it's slow, only to find that the parallelized version isn't that much faster (my computer is just hotter and louder), and then giving in and rewriting the code in C++.

Re: What's up, Python? The GIL removed, a new compiler, optparse deprecated

#195

Post by tgv » Mon, Jul 31, 2023, 8:15 AM UTC

zarzavat wrote: post #131

Jtsummers wrote: post #10

Earlier quoted context omitted.

That discussion was amusing. Removing the GIL opens up the possibility of actually getting a real performance benefit from multithreaded Python code. That's the value. Given every modern desktop and server is multicore (and increasingly getting to tens of cores if not hundreds), multithreading in Python unhampered by the GIL will be a useful thing. And no, multiprocessing is not a good alternative to multithreading.…

Python is not a language for writing fast code. Python is a relaxed language for things that don’t have to be fast. If you need something to be fast you are supposed to use a C extension and control it with Python - that’s been the dogma for as long as I can remember to avoid exactly this kind of pathological race to performance in a language that was never designed for it. By using Python you are already leaving a t…

Yup. I don't know why you would insist on having multiple Python threads, especially given the high risks. Python is only suitable for coordinating/scripting large libraries written in other languages or for quick and easy development. Python programs should not reach the stage where their use in production is hampered by lack of multi-threading.

Re: What's up, Python? The GIL removed, a new compiler, optparse deprecated

#196

Post by tgv » Mon, Jul 31, 2023, 8:20 AM UTC

School-Cotton wrote: post #15

masklinn wrote: post #9

Earlier quoted context omitted.

Even then the GIL can cause issues, concerns of PyTorch are specifically one of the motivations of the PEP, and one of the reasons Meta / FB really really wants this: > In PyTorch, Python is commonly used to orchestrate ~8 GPUs and ~64 CPU threads, growing to 4k GPUs and 32k CPU threads for big models. While the heavy lifting is done outside of Python, the speed of GPUs makes even just the orchestration in Python not…

I feel like orchestrating thousands of GPUs is such a niche use case that it’s fair to expect the people wanting to do it to learn a more suited language, rather than ruining Python for everyone else.

You are completely right. Why don't they write their stuff in another language? They've got the resources. Now the rest of the world will suffer the consequences, one of which may be that the devs of native libs will simply abandon the work, or that those libs will become too difficult to use for the casual or starting programmer, completely defeating the purpose.

I'm fine with two builds, but not a single non-GIL build.

Re: What's up, Python? The GIL removed, a new compiler, optparse deprecated

#197

Post by Jiocus » Mon, Jul 31, 2023, 8:24 AM UTC

wtetzner wrote: post #142

dragonwriter wrote: post #136

Earlier quoted context omitted.

Because at the time of the 2-3 migration, parallelism wasn’t viewed as being as important as it is today.

Is that really true? We already had multicore machines, and Herb Sutter's "The free lunch is over" article had been published for years by then.

I'm sure parallelism was understood and worked towards in computing at large and within certain programming languages. The then contemporary and popular use-cases for Python (which were they?) might have had very different challenges solved by other means.

(I'm just guessing here)

Re: What's up, Python? The GIL removed, a new compiler, optparse deprecated

#198

Post by BiteCode_dev » Mon, Jul 31, 2023, 8:25 AM UTC

schemescape wrote: post #51

The title says "GIL removed", but the article says "This means in the coming years, Python will have its GIL removed." I'm assuming the article is correct and the GIL has not been removed yet (but there is a plan to remove it in the future). If that's not the case, please correct me!

Yes.

I tried to come up with something that would convey in a few words that the GIL was going to be removed for sure this time. But as a Frenchmen, I couldn't find better.

"GIL will be removed" was the closest, but it's very long, and it sounds like all those times we had the promise it would be, but it never did.

So the Prophetic perfect tense is the best compromise: it asserts near certainty, it's short, and worst case scenario the article remove ambiguity.

Plus the news popped up this week in HN front page, so a lot of people knew the context.

Re: What's up, Python? The GIL removed, a new compiler, optparse deprecated

#199

Post by tgv » Mon, Jul 31, 2023, 8:25 AM UTC

quickthrower2 wrote: post #27

csmpltn wrote: post #17

Earlier quoted context omitted.

There are plenty of other Python VMs that don't have a GIL and can be used already today, out of the box (examples include Jython and IronPython). Despite that fact - CPython remains the most popular Python VM out there (it utilizes a GIL). Instead of waiting for the GIL to be removed out of CPython - take your fancy Python code and just run it using a different VM. It's literally as simple as that. If the GIL was su…

Would a large codebase seemlessly run on another interpreter?

I think you underestimate the problems that will occur in a large code base when the GIL is gone. It'll play out like this:

Test will be fine, but production will have some weird bugs. Nobody understands it. The devs end up adding locks everywhere, bringing down performance, or creating dead locks. In the end, they migrate back to Python 3.16.

Here's free lesson number 1: start adding stress tests now.

Re: What's up, Python? The GIL removed, a new compiler, optparse deprecated

#200

Post by knorker » Mon, Jul 31, 2023, 8:26 AM UTC

Waterluvian wrote: post #32

wmwmwm wrote: post #14

Historically I’ve written several services that load up some big datastructure (10s or 100s of GB), then expose an HTTP API on top of it. Every time I’ve done a quick implementation in Python of a service that then became popular (within a firm, so 100s or 1000s of clients) I’ve often ended up having to rewrite in Java so I can throw more threads at servicing the requests (often CPU heavy). I may have missed somethin…

No, that’s about right. The response, which isn’t technically wrong, is “unless you’re CPU bound, your application should be parallized with a WSGI. You shouldn’t be loading all that up in memory so it shouldn’t matter that you run 5 Python processes that each handle many many concurrent I/O bound requests.” And this is kinda true… I’ve done it a lot. But it’s very inflexible. I hate programming architectures/pattern…

Well, it's like showing your plan for painting a room, and asking "I seem to get stuck here after painting all but the corner, how do I get out of the corner?". The answer actually is "don't leave the corner for last".

Or like the martial arts student asking the master "how do I fight a guy 100m away with a rifle?" - "don't be there".