Earlier quoted context omitted.
I do so love playing guessing games for every comment I read.
I don't think it's reasonable to expect people here to cater to a least common denominator of comprehension.
Intent to approve PEP 703: making the GIL optional
431–440 of 513 posts
Re: Intent to approve PEP 703: making the GIL optional
#432Earlier quoted context omitted.
I do so love playing guessing games for every comment I read.
I don't think it's reasonable to expect people here to cater to a least common denominator of comprehension.
I suppose I could just dismiss the comment, but curiosity got the better part of me. I'm glad someone asked. I also had to search what "4GL" means. I'm assuming this top Wikipedia hit [1] is what the OP meant.
I don't know if it's reasonable to expect people to know these terms. But, the concise text is not saving anyone beyond the OP any appreciable amount of time, so what's its value?
[1] https://en.m.wikipedia.org/wiki/Fourth-generation_programmin...
Re: Intent to approve PEP 703: making the GIL optional
#433Earlier quoted context omitted.
I do so love playing guessing games for every comment I read.
I don't think it's reasonable to expect people here to cater to a least common denominator of comprehension.
Re: Intent to approve PEP 703: making the GIL optional
#434Earlier quoted context omitted.
I work at a big company. We had probably millions of lines of Python. We migrated to c++ instead of to Python 3.
I'm really struggling to think of a scenario where that makes sense
Re: Intent to approve PEP 703: making the GIL optional
#435Earlier quoted context omitted.
That Java had concurrency built in from the start is a blessing mostly, but also a bit of a curse. Most of the Java ecosystem is still in the mindset that threads are cheap and firing up a couple more cannot hurt. So we end up with apps that run thousands of threads and this disease is hard to contain.
Like not everyone toy app isn't going to be the next FAANG, there are plenty of workloads where it hardly matters, while 30 years later it is still a mess in C and C++. And between C++ and Rust coroutines, still not sure which one I like less.
Re: Intent to approve PEP 703: making the GIL optional
#436Earlier quoted context omitted.
From PEP 703: > Manuel Kroiss, software engineer at DeepMind on the reinforcement learning team, describes how the bottlenecks posed by the GIL lead to rewriting Python codebases in C++, making the code less accessible: > "We frequently battle issues with the Python GIL at DeepMind. In many of our applications, we would like to run on the order of 50-100 threads per process. However, we often see that even with fewer…
So Python is being fundamentally changed for everyone because of the needs of a niche subset of Python programmers (AI researchers), because that niche subset refuses to learn a language more suited to their task?
Re: Intent to approve PEP 703: making the GIL optional
#437Earlier quoted context omitted.
Like not everyone toy app isn't going to be the next FAANG, there are plenty of workloads where it hardly matters, while 30 years later it is still a mess in C and C++. And between C++ and Rust coroutines, still not sure which one I like less.
I thought so too until I got to interact with databases and Big Data tools written in Java. God, what a mess that requires so much upkeeping, more dependency problems than I remember from C++ and probably some orders of magnitude more resources than they should.
Re: Intent to approve PEP 703: making the GIL optional
#438Earlier quoted context omitted.
PEP-703 contains a whore Motivation section. Long enough to require a summary: > Python’s global interpreter lock makes it difficult to use modern multi-core CPUs efficiently for many scientific and numeric computing applications. Heinrich Kuttler, Manuel Kroiss, and Paweł Jurgielewicz found that multi-threaded implementations in Python did not scale well for their tasks and that using multiple processes was not a su…
Your comment has been nominated for the best typo in 2023. Let's hope the change is not this badly mercenary.
Re: Intent to approve PEP 703: making the GIL optional
#439Earlier quoted context omitted.
> The fact of the matter is that Python (already today) allows you to achieve parallelism across both IO-bound and CPU-bound workloads. I also don't need to use goroutines. I could simply spin up my golang application as a couple of processes, and use pipes and other IPC to coordinate them. "There is another way to do X" doesn't imply that other way is better.
> "doesn't imply that other way is better" I never said that multiprocessing is "better" than multithreading. Both are mechanisms to achieve parallelism with different pros/cons, and there are legit cases where threads are a necessity that can't be satisfied with processes (ref my previous comment with examples). This conversation would've been much easier to have given specific constraints and examples (which nobody…
How do you figure that? Even python2 already supported the usage of OS threads [1].
> 30+ years' worth of 3rd-party packages and libraries which were never built with multithreading in-mind
Many of these packages also don't use other forms of concurrency, but are simply encapsulated functionality that runs in a single thread. Meaning, they will not be bothered by the change.
Besides, as I have mentioned elsewhere, library maintainers always need to keep up with the development of the underlying language as well as usage patterns of the community, or their libraries become obsolete. That is true no matter what programming language we talk about.
> If you need hardcore, ultra efficient and parallelized workflows - Python is just the wrong tool for the job period
Python is already used as an orchestration language for huge numerical workloads, be it data science or machine learning. It is simple, intuitive and has by far the largest library support of any contemporary language.
There is simply no good argument, why the language that we entrust to orchestrate this scale of computing power, shouldn't itself be as efficient as possible for a dynamically typed script-language. That this is absolutely possible, is demonstrated by languages like Julia.
The fact that Python will never be as fast as Go, Rust or C++, doesn't change that.
> Those people want to get shit done quickly - and mutexes, sempahores, events, threads, synchronization primitives, atomics, etc - will do nothing but make their lives a misery, and drive them away.
Those people will for the most not even realize that the GIL is gone. If they write...
- single threaded synchronous code
- asyncio based code
- multiprocessing code
...the change doesn't matter to them. The hobbyists small webserver, or the medium companies Flask-based webapp will still run as before. And if they write threading code, and do so correctly, then it is very likely the only change they will see, is that suddenly their application runs faster under high load.The removal of the GIL neither takes away existing capabilities from Python, nor does it force everyone to write threading code.
> and you're not really responding which makes me feel like we're not conversing here...
That's because I have done so elsewhere in this thread already [2]
Re: Intent to approve PEP 703: making the GIL optional
#440Unpopular opinion: This is a missed opportunity. What? Python could have been the one language with a sane multithreading model. Now it risks becoming a second version of Java. I fear this will make it a less attractive programming language, not least because it might lose its beginner friendlyness. For example, without the GIL a lot more care must be put into designing your programs. This can be true even though you…
I’m genuinely curious, What's wrong with Java's model?
* Threads are also relatively fat. You'd probably think twice about spinning up 1000 of them.
* Futures came in with Java 8 and are lighter weight, but have certain implementation defects - can't cancel them for example. Although there are some instances in which regular threads aren't cancellable. Also, the API isn't terribly popular. I happen to like flatmapping my way around a codebase, but plenty of devs' eyes glaze over when they read that code.
* Fibers are released I think - at least in beta. More like JS's model I believe. They're even lighter weight and get rid of the flatmappy stuff again. I assume it's still up to the programmer to get the locking right.