Let's Remove the Global Interpreter Lock

Hi, blog post author here. Let me put an offer here:

If you want to ask a question that warrants a response (as opposed to promoting your own effort, which is valid but does not warrant a response), please mail me, the mail is public and I'll put the responses publically on either my blog or pypy blog.

If you have long running, computationally intensive code, with simple interactions, sure. Then multiple processes is the right thing.

But sometimes you are writing a GUI app, or some "real time" code [1]. You put blocking calls onto a different thread to keep the UI responsive. But then you find that still the blocking calls freeze the UI across threads due to the GIL.

Pure Python code is not the problem in this case - the GIL gets released between statements often enough. It is long running C code. You could release the GIL manually in there, but it is not done everywhere. Also, there are often calls that are supposed to be instant (like opening a file, or starting an async operation), that take seconds under bad conditions (when the network is down).

----

[1] well, with Python probably not in the strict definition of real time, but say you are controlling some external device

> Is Python really any different or is it just wishful thinking?

I'll stick my neck out and say that personally I feel that Python is actually better in this regard. Perl pissed me off so much that I left Perl at the height of its popularity. I probably had least 5 years(probably closer to 10+, but my memory of dates is fuzzy--pretty sure I never used Perl 3, though ...) years of professional use of Perl under my belt at that point.

I still migrated to Python.

That's a pretty big indictment when someone is willing to go back to being a n00b rather than continue to put up with continuing grief.

> Why do I see Python programs that look like a weird mix of Lisp and Java? Surely it is because even with what Python enforce, there are many many ways to produce unclear code that really don't even has to do with the language used.

The difference, from my experience, is that newbies in Python don't write code that A) other newbies can't understand and B) requires excessive mental effort from experienced folks to understand. Neither of those were true in my experience for Perl. A newb in Perl would very quickly trip something that would boggle even your local Perl expert.

I remember when I was a beginner and posted a 15 line program to comp.lang.perl and watched several of the experts actually wondering what the correct interpretation of the grammar was. I don't think I ever managed to cause something like that in Python. Obviously, I threw that program out post haste.

Sure, people can make amazingly complicated things with generators, decorators, etc once they get the hang of the language. However, newbies don't normally do this in Python. As you point out, they normally write it like Java (for better and worse).

In Perl, newbies immediately had to grapple with things like list vs scalar context and god help you if you tripped over a corner case (although, to be fair, God, in the form Randall Schwartz, WOULD quite often help you if you asked on comp.lang.perl ...)

> And why did my employer see the need for the comprehensive Python style guidelines manual... I guess Python bit just as hard as Perl.

I would personally expect that any company writing a very large quantity of code would produce a style guide for any language they use.

> The quality of Perl frameworks, libs, example code etc is actually increasing and getting easier to find.

That's actually really cool.

It is possible to do more after a fork (cf. async-signal-safe), but it's hairy enough to just say — don't, always exec (similar to how doing actual work in a signal handler is generally a very bad idea).

I still wish to find a good tutorial about memmap. The doc about it is very formal. Something with clear use cases, patterns, gotchas and best practice would probably make it more popular.