Live data from Hacker News

Intent to approve PEP 703: making the GIL optional

discuss.python.org

431–440 of 513 posts

Re: Intent to approve PEP 703: making the GIL optional

#431

Earlier quoted context omitted.

I do so love playing guessing games for every comment I read.

I don't think it's reasonable to expect people here to cater to a least common denominator of comprehension.

Nothing “least common denominator of comprehension” about not knowing what that acronym means. I also didn’t know.

Re: Intent to approve PEP 703: making the GIL optional

#432

Earlier quoted context omitted.

I do so love playing guessing games for every comment I read.

I don't think it's reasonable to expect people here to cater to a least common denominator of comprehension.

That's fine. But, communication is a skill and like all skills, can be improved. For my part, this is the first time I can remember seeing MIS used in the wild (i.e., not in university). Given the tech industry's propensity for overloading initialisms and redefining terms, I have to go by context to work out the meaning. In this case, the context was insufficient.

I suppose I could just dismiss the comment, but curiosity got the better part of me. I'm glad someone asked. I also had to search what "4GL" means. I'm assuming this top Wikipedia hit [1] is what the OP meant.

I don't know if it's reasonable to expect people to know these terms. But, the concise text is not saving anyone beyond the OP any appreciable amount of time, so what's its value?

[1] https://en.m.wikipedia.org/wiki/Fourth-generation_programmin...

Re: Intent to approve PEP 703: making the GIL optional

#433

Earlier quoted context omitted.

I do so love playing guessing games for every comment I read.

I don't think it's reasonable to expect people here to cater to a least common denominator of comprehension.

Yes but unless I'm a extreme outlier "MIS" is not part of the common denominator.

Re: Intent to approve PEP 703: making the GIL optional

#434

Earlier quoted context omitted.

I work at a big company. We had probably millions of lines of Python. We migrated to c++ instead of to Python 3.

I'm really struggling to think of a scenario where that makes sense

I'm not convinced it's was the right choice either, but I'm not keen on Python, and not sold on Rust. Maybe Zig? I don't know. I personally like Typescript but even I will admit I'm not sure it's the best choice for a large, server-centric company.

Re: Intent to approve PEP 703: making the GIL optional

#435
post #309

Earlier quoted context omitted.

That Java had concurrency built in from the start is a blessing mostly, but also a bit of a curse. Most of the Java ecosystem is still in the mindset that threads are cheap and firing up a couple more cannot hurt. So we end up with apps that run thousands of threads and this disease is hard to contain.

Like not everyone toy app isn't going to be the next FAANG, there are plenty of workloads where it hardly matters, while 30 years later it is still a mess in C and C++. And between C++ and Rust coroutines, still not sure which one I like less.

I thought so too until I got to interact with databases and Big Data tools written in Java. God, what a mess that requires so much upkeeping, more dependency problems than I remember from C++ and probably some orders of magnitude more resources than they should.

Re: Intent to approve PEP 703: making the GIL optional

#436
post #399

Earlier quoted context omitted.

From PEP 703: > Manuel Kroiss, software engineer at DeepMind on the reinforcement learning team, describes how the bottlenecks posed by the GIL lead to rewriting Python codebases in C++, making the code less accessible: > "We frequently battle issues with the Python GIL at DeepMind. In many of our applications, we would like to run on the order of 50-100 threads per process. However, we often see that even with fewer…

So Python is being fundamentally changed for everyone because of the needs of a niche subset of Python programmers (AI researchers), because that niche subset refuses to learn a language more suited to their task?

In a word - nope!

Re: Intent to approve PEP 703: making the GIL optional

#437
post #309

Earlier quoted context omitted.

Like not everyone toy app isn't going to be the next FAANG, there are plenty of workloads where it hardly matters, while 30 years later it is still a mess in C and C++. And between C++ and Rust coroutines, still not sure which one I like less.

I thought so too until I got to interact with databases and Big Data tools written in Java. God, what a mess that requires so much upkeeping, more dependency problems than I remember from C++ and probably some orders of magnitude more resources than they should.

Doing code review for C++ code delivered by most well know offshoring companies, versus what they deliver in Java, will help get another point of view.

Re: Intent to approve PEP 703: making the GIL optional

#438
post #408
post #386

Earlier quoted context omitted.

PEP-703 contains a whore Motivation section. Long enough to require a summary: > Python’s global interpreter lock makes it difficult to use modern multi-core CPUs efficiently for many scientific and numeric computing applications. Heinrich Kuttler, Manuel Kroiss, and Paweł Jurgielewicz found that multi-threaded implementations in Python did not scale well for their tasks and that using multiple processes was not a su…

Your comment has been nominated for the best typo in 2023. Let's hope the change is not this badly mercenary.

What?

Re: Intent to approve PEP 703: making the GIL optional

#439

Earlier quoted context omitted.

> The fact of the matter is that Python (already today) allows you to achieve parallelism across both IO-bound and CPU-bound workloads. I also don't need to use goroutines. I could simply spin up my golang application as a couple of processes, and use pipes and other IPC to coordinate them. "There is another way to do X" doesn't imply that other way is better.

> "doesn't imply that other way is better" I never said that multiprocessing is "better" than multithreading. Both are mechanisms to achieve parallelism with different pros/cons, and there are legit cases where threads are a necessity that can't be satisfied with processes (ref my previous comment with examples). This conversation would've been much easier to have given specific constraints and examples (which nobody…

> Within the context of a general purpose VM which was never built to support multithreading (CPython),

How do you figure that? Even python2 already supported the usage of OS threads [1].

> 30+ years' worth of 3rd-party packages and libraries which were never built with multithreading in-mind

Many of these packages also don't use other forms of concurrency, but are simply encapsulated functionality that runs in a single thread. Meaning, they will not be bothered by the change.

Besides, as I have mentioned elsewhere, library maintainers always need to keep up with the development of the underlying language as well as usage patterns of the community, or their libraries become obsolete. That is true no matter what programming language we talk about.

> If you need hardcore, ultra efficient and parallelized workflows - Python is just the wrong tool for the job period

Python is already used as an orchestration language for huge numerical workloads, be it data science or machine learning. It is simple, intuitive and has by far the largest library support of any contemporary language.

There is simply no good argument, why the language that we entrust to orchestrate this scale of computing power, shouldn't itself be as efficient as possible for a dynamically typed script-language. That this is absolutely possible, is demonstrated by languages like Julia.

The fact that Python will never be as fast as Go, Rust or C++, doesn't change that.

> Those people want to get shit done quickly - and mutexes, sempahores, events, threads, synchronization primitives, atomics, etc - will do nothing but make their lives a misery, and drive them away.

Those people will for the most not even realize that the GIL is gone. If they write...

    - single threaded synchronous code 
    - asyncio based code
    - multiprocessing code
...the change doesn't matter to them. The hobbyists small webserver, or the medium companies Flask-based webapp will still run as before. And if they write threading code, and do so correctly, then it is very likely the only change they will see, is that suddenly their application runs faster under high load.

The removal of the GIL neither takes away existing capabilities from Python, nor does it force everyone to write threading code.

> and you're not really responding which makes me feel like we're not conversing here...

That's because I have done so elsewhere in this thread already [2]

[1]: https://docs.python.org/2/library/threading.html

[2]: https://news.ycombinator.com/item?id=36918250

Re: Intent to approve PEP 703: making the GIL optional

#440
post #169

Unpopular opinion: This is a missed opportunity. What? Python could have been the one language with a sane multithreading model. Now it risks becoming a second version of Java. I fear this will make it a less attractive programming language, not least because it might lose its beginner friendlyness. For example, without the GIL a lot more care must be put into designing your programs. This can be true even though you…

I’m genuinely curious, What's wrong with Java's model?

* It's hard to program correctly using the thread model. And saying that locks are the solution to thread safety issues in Java, is like saying that malloc/free is the solution to memory safety issues in C.

* Threads are also relatively fat. You'd probably think twice about spinning up 1000 of them.

* Futures came in with Java 8 and are lighter weight, but have certain implementation defects - can't cancel them for example. Although there are some instances in which regular threads aren't cancellable. Also, the API isn't terribly popular. I happen to like flatmapping my way around a codebase, but plenty of devs' eyes glaze over when they read that code.

* Fibers are released I think - at least in beta. More like JS's model I believe. They're even lighter weight and get rid of the flatmappy stuff again. I assume it's still up to the programmer to get the locking right.

Post reply on HN