The GIL in an implementation detail of CPython, it's not part of Python the language.
It's been a while since I've followed Python closely but last I heard Python-the-language is de facto defined as "whatever CPython does". Has Python since grown a proper language specification?
Python doesn't have a full specification, but it does have first party documentation that distinguishes between CPython implementation details and language guarantees, in part to support alternative implementations.
This specification does not mention the GIL anywhere, which means it is, as the GP said, an implementation detail. Other implementations of Python that do not have the GIL are still "Python" implementations because they meet this language specification.
It seems like in every python discussion I hear people complain about the GIL. I’m happy people are working on removing the GIL, but As a professional python dev for about 5 years now I have literally never had a problem where the GIL was a limiter. Although I just make web apps so maybe I’m not the target audience.
Conversely, I've worked on backend, data processing-type applications for most of my career, much of it in Java but some (especially recently) in Python, and the GIL is a huge limiting factor for writing efficient, readable Python code. I've had to write very annoying Python code using the multiprocessing library to get around the GIL, and ultimately it works, but it's ugly and clunky and overall just a pain. And remember, I've written a lot of Java code, so I have a high tolerance for pain! But the JVM's concurrency abstractions are actually kind of a joy to use, even if Java the language isn't. Python is the opposite, so if they can shed the GIL and make multithreading viable in Python without forking new processes, that would be a huge win.
It's been a while since I've followed Python closely but last I heard Python-the-language is de facto defined as "whatever CPython does". Has Python since grown a proper language specification?
Python doesn't have a full specification, but it does have first party documentation that distinguishes between CPython implementation details and language guarantees, in part to support alternative implementations.
Code that assumes that something is going to be atomic because of the GIL (or any other implementation detail) is simply broken. If you need something to be atomic you should be explicit about that and use mutex or something.
I dunno about that. There's a lot of Python out there with worker threads dumping their output into a shared "results" dict on the assumption that those insert operations are atomic.
For example, the following operations are all atomic (L, L1, L2
are lists, D, D1, D2 are dicts, x, y are objects, i, j are ints):
L.append(x)
L1.extend(L2)
x = L[i]
x = L.pop()
L1[i:j] = L2
L.sort()
x = y
x.field = y
D[x] = y
D1.update(D2)
D.keys()
Python doesn't have a full specification, but it does have first party documentation that distinguishes between CPython implementation details and language guarantees, in part to support alternative implementations.
> Python doesn't have a full specification This doesn't count? https://docs.python.org/3/reference/index.html
The introduction section states that it's not a complete/exact specification. If you're using a Python implementation and have a question, this should answer it. If you're writing a Python implementation and have a question, this may not answer it.
> Consequently, if you were coming from Mars and tried to re-implement Python from this document alone, you might have to guess things and in fact you would probably end up implementing quite a different language. On the other hand, if you are using Python and wonder what the precise rules about a particular area of the language are, you should definitely be able to find them here. If you would like to see a more formal definition of the language, maybe you could volunteer your time — or invent a cloning machine :-).
It seems like in every python discussion I hear people complain about the GIL. I’m happy people are working on removing the GIL, but As a professional python dev for about 5 years now I have literally never had a problem where the GIL was a limiter. Although I just make web apps so maybe I’m not the target audience.
It's just about what kind of code you write.
If you write a lot of code that parallelizes over data you will hurt all the time because of the GIL.
If you have worker processes that do something on the CPU, and the results need to be collected and processed further in some other process, you now need to pickle the data to copy it around. That can get really slow.
I understand a lot of python users don't do that kind of thing, but it's a real problem. I'm happy that the python community seems to be slowly beginning to take this seriously after decades of just claiming the GIL isn't a real issue.
Python doesn't have a full specification, but it does have first party documentation that distinguishes between CPython implementation details and language guarantees, in part to support alternative implementations.
> Python doesn't have a full specification This doesn't count? https://docs.python.org/3/reference/index.html
I would say that that is intended as documentation for users (includong implementors of tools targeting the language), not a specification for implementors of Python, but I would agree that it is largely usable in either role; my point in the GP was that something distinct called “the Python Language Specification” doesn't exist, but that a (not necessarily complete, from a language implementors perspective) specification distinct from the implementation behavior of CPython does effectively exist in the documentation.
Is there any versioning scheme that is comparable? Honest question.
Some languages just use integer versions, like Java.
Java dropped the leading 1 with version 5 since Sun decided it would never go for a complete rewrite of the language. Imagine the chaos if Python did that and then pulled the 2 to 3 change on a run of the mill update from 213 to 214.