Although this is nice, the problems with the GIL are often blown out of proportion: people stating that you couldn't do efficient (compute-bounded) multi-processing, which was never the case as the `multiprocessing` module works just fine.
> as the `multiprocessing` module works just fine.
Something that tripped me up when I last did `multiprocessing` was that communication between the processes requires marshaling all the data into a binary format to be unmarshaled on the other side; if you're dealing with 100s of MB of data or more, that can be quite some significant expense.
I've been programming in Python for over 6 years now and every week I learn something new. But recently I've been thinking about moving to a more capable language with proper concurrency for backend API requests (FastAPI sucked). I also want types, so Elixir is not in the picture. I dabbled in Rust a bit. Although I was able to get the hang of things and build a CLI tool pretty quickly, I'm worried I'll have to deal…
Swift or Kotlin might be what you’re looking for but nobody uses Swift for backend really, and I’m not sure about Kotlin.
Kotlin has basically replaced java for many spring shops. It's really common in backend nowadays.
Well this release will break any code that uses threads. The goal of this particular release is to work for thread-free programs.
How do single-threaded programs benefit from a lack of GIL?
Speed. Admittedly not quite as much so the way this patch is implemented, since it just short circuits the extra function calls, doesn’t omit them entirely.
And I guess what I don't understand is why people choose Python for these use cases. I am not in the "Rustify" everything camp, but Go + C, Java + JNI, Rust, and C++ all seem like more suitable solutions.
Notably, all of those are static languages and none of them have array types as nice as PyTorch or NumPy, among many other packages in the Python ecosystem. Those two facts are likely closely related.
If only there were a dynamic language which performs comparably to C and Fortran, and was specifically designed to have excellent array processing facilities.
Unfortunately, the closest thing we have to that is Julia, which fails to meet none of the requirements. Alas.
I've been programming in Python for over 6 years now and every week I learn something new. But recently I've been thinking about moving to a more capable language with proper concurrency for backend API requests (FastAPI sucked). I also want types, so Elixir is not in the picture. I dabbled in Rust a bit. Although I was able to get the hang of things and build a CLI tool pretty quickly, I'm worried I'll have to deal…
Sounds like you want Julia. It looks like Python, but also has, what you ask for. You can even run Python from Julia, so that alleviates the problem with a lack of libraries somewhat.
How do single-threaded programs benefit from a lack of GIL?
Disabling the GIL can unlock true multi-core parallelism for multi-threaded programs, but this requires code to be restructured for safe concurrency, which isn't that difficult it seems: > When we found out about the “nogil” fork of Python it took a single person less than half a working day to adjust the codebase to use this fork and the results were astonishing. Now we can focus on data acquisition system developme…
>We frequently battle issues with the Python GIL at DeepMind. In many of our applications, we would like to run on the order of 50-100 threads per process. However, we often see that even with fewer than 10 threads the GIL becomes the bottleneck. To work around this problem, we sometimes use subprocesses, but in many cases the inter-process communication becomes too big of an overhead. To deal with the GIL, we usually end up translating large parts of our Python codebase into C++. This is undesirable because it makes the code less accessible to researchers.
Well this release will break any code that uses threads. The goal of this particular release is to work for thread-free programs.
This isn't correct. TFA said that small threaded programs had been run successfully, but that the test suite broke in asyncio. Async I/O and threads are two different things, and either can be present in real code without the other.
"small threaded programs had been run successfully"
I have ran a lot of programs containing race conditions successfully many times until I ran into an issue.
While the title is correct, it is a bit misleading, because disabling the GIL breaks the asyncio tests. It's like saying the engine can be removed from my car. Sure, it can, but the car won't work.
Being able to remove the engine from my car with the push of a button would be a pretty amazing feature!