The Changing "Guarantees" Given by Python's Global Interpreter Lock • Hacker News Board

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#121

Post by Kranar » Fri, Nov 17, 2023, 7:39 PM UTC

pdonis wrote: post #119

Kranar wrote: post #117

Earlier quoted context omitted.

I never mentioned any advantage or disadvantage. I am only stating a fact about the current state of Python. Python does not have a specification or a standards document, it has a reference that describes how Python happens to work and the reference is a great resource for people to familiarize themselves with the language but it should simply be clear that its purpose is to reflect the existing state of the language…

Then why are different implementations of Python all implementations of Python? What makes them implementations of Python rather than different languages? I've already given my answer: they all meet the language reference I linked to (yes, the word "reference" appears in its title, not "specification"; that's just another quibble). What is your answer?

Haha, no they don't all meet the language reference you linked to and if you used them you'd know that!

Python is a lot more like LISP than it is C++. There are many flavors of Python, from MicroPython to GraalPython to Cyston and literally dozens of them. They most certainly do not all meet the language reference you linked to and they all have quirks here and there.

A language does not need a specification in order to exist or to have a name. What matters is that people use it and get work done with it, and ultimately if looks like a Python and quacks like a Python, then it's fine to call it a Python.

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#122

Post by Kranar » Fri, Nov 17, 2023, 7:42 PM UTC

pdonis wrote: post #120

Kranar wrote: post #118

Earlier quoted context omitted.

Plenty of languages have a specification. C++, Java, C# all do. Some languages do not, such as Python and Rust. The ISO C++ committee even uses quite strong language about the C++ standard and makes it a point to differentiate between the C++ specification and C++ references: >The standard is not intended to teach how to use C++. Rather, it is an international treaty – a formal, legal, and sometimes mind-numbingly de…

Does every C++ implementation do exactly what is in the C++ language specification, no more, no less? Same question for Java and C#. If the answer to all of these questions is "no", as I believe it is, on what grounds do you claim that these languages have a specification, while Python and Rust do not?

Yes the C++ and Java implementations do exactly what is in the language specification. The C++ specification explicitly allows languages to do more, but it can not do less. I believe Java has a similar clause but I'm not sure.

The grounds that I claim is that you can read them, here they are:

https://docs.oracle.com/javase/specs/

https://isocpp.org/files/papers/N4860.pdf

Note what the actual C++ standard states, and I quote:

>This document specifies requirements for implementations of the C++ programming language. The first such requirement is that they implement the language, so this document also defines C++. Other requirements and relaxations of the first requirement appear at various places within this document

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#123

Post by samus » Fri, Nov 17, 2023, 8:02 PM UTC

Kranar wrote: post #77

pdonis wrote: post #52

Earlier quoted context omitted.

> AFAIK, there is no Python language specification Yes, there is: https://docs.python.org/3/reference/index.html This specification does not mention the GIL anywhere, which means it is, as the GP said, an implementation detail. Other implementations of Python that do not have the GIL are still "Python" implementations because they meet this language specification.

That is not a specification. A specification is a prescriptive document of how a system is required to work whereas a reference is a descriptive document of how a system happens to currently work. Python has a reference, in fact it has many references that even contradict one another in subtle ways, but it does not have a specification.

Whether it is a specification or a reference doesn't really matter for this discussion. What matters is that it is the closest thing to a specification that exists for Python. Assuming anything beyond it ties the program to internals of a specific implementation. Internals for which the vendor might have different intentions about keeping stable or change than users expect.

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#124

Post by eesmith » Fri, Nov 17, 2023, 8:06 PM UTC

Kranar wrote: post #117

eesmith wrote: post #107

Earlier quoted context omitted.

What advantages would there be to that document? I mean, I can see the point if there are multiple commercial competitors in the market, as there is with C/C++, or if the implementation is proprietary and the users want to avoid vendor lock-in. But the Minimal BASIC of ANSI X3.60-1978 never did catch on for any of the BASICs I used in the 1990s, and the Full BASIC of ANSI X3.113-1987 was a flop, so clearly it's possi…

I never mentioned any advantage or disadvantage. I am only stating a fact about the current state of Python. Python does not have a specification or a standards document, it has a reference that describes how Python happens to work and the reference is a great resource for people to familiarize themselves with the language but it should simply be clear that its purpose is to reflect the existing state of the language…

What then did you think it relevant to point out that Python does not have a prescriptive document if you didn't think it was somehow useful?

Here's another fact: Python doesn't have its own typeface either. Yet that fact is hardly germane to the thread.

A prescriptive document is a formal specification. The Python Language Reference is a less formal specification. It's still a specification.

It is also not a single document, but then again even a formal specification may incorporate other specifications by reference.

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#125

Post by samus » Fri, Nov 17, 2023, 8:08 PM UTC

Kranar wrote: post #118

pdonis wrote: post #116

Earlier quoted context omitted.

> That is not a specification. You're quibbling. By your definition, no programming language has a "specification". Every language has implementations that do things that aren't explicitly described in any document.

Plenty of languages have a specification. C++, Java, C# all do. Some languages do not, such as Python and Rust. The ISO C++ committee even uses quite strong language about the C++ standard and makes it a point to differentiate between the C++ specification and C++ references: >The standard is not intended to teach how to use C++. Rather, it is an international treaty – a formal, legal, and sometimes mind-numbingly de…

The C and C++ specifications are rather infamous about being incomplete, i.e., containing features that might lead to undefined behavior. On the other hand, Java is quite comprehensive. It is one of the earliest instances of languages specifying a memory model.

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#126

Post by luhn » Fri, Nov 17, 2023, 8:12 PM UTC

Aurornis wrote: post #13

coldcode wrote: post #4

Why does Python use an un-comparable version number scheme? Not being a Python programmer, comparing version 3.9 to 3.13 seemed bizarre until I caught on.

Semantic Versioning is ubiquitous across the modern software industry. It’s worth reading about it if you’re not familiar: https://semver.org/ Never assume that version numbers are decimal values. It’s more obvious when you see the full version triple (3.13.0 for example) that it’s not a single number, but the abbreviated version numbers can some times look like a decimal value. You should never compare version numbe…

I know you didn't say it outright, but since it is implied: Python does not use semver, although it does share similar formatting. Semver considers bumps to the second integer a "minor" release with no breaking changes, whereas in Python that indicates a major release that is not backwards compatible.

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#127

Post by Kranar » Fri, Nov 17, 2023, 8:14 PM UTC

eesmith wrote: post #124

Kranar wrote: post #117

Earlier quoted context omitted.

I never mentioned any advantage or disadvantage. I am only stating a fact about the current state of Python. Python does not have a specification or a standards document, it has a reference that describes how Python happens to work and the reference is a great resource for people to familiarize themselves with the language but it should simply be clear that its purpose is to reflect the existing state of the language…

What then did you think it relevant to point out that Python does not have a prescriptive document if you didn't think it was somehow useful? Here's another fact: Python doesn't have its own typeface either. Yet that fact is hardly germane to the thread. A prescriptive document is a formal specification. The Python Language Reference is a less formal specification. It's still a specification. It is also not a single…

Because the original comment was about the appropriateness of writing Python code according to the reference implementation as opposed to writing Python code strictly according to the reference document. Some people are arguing that the reference documentation is in some sense authoritative and the only resource that should be used to define Python's semantics. In particular the issue at hand is whether the GIL is a part of the semantics of a Python program.

As my position is that the CPython implementation is the reference implementation for Python and as the GIL is an integral part of that implementation, then the GIL does form a part of Python's semantics regardless of whether the reference documentation mentions it or not.

The Python reference documentation is not a specification and it's not intended to be one.

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#128

Post by samus » Fri, Nov 17, 2023, 8:16 PM UTC

bheadmaster wrote: post #3

dfox wrote: post #2

Code that assumes that something is going to be atomic because of the GIL (or any other implementation detail) is simply broken. If you need something to be atomic you should be explicit about that and use mutex or something.

True, but when a majority of ecosystem is relying on an implementation detail, that implementation detail becomes de-facto standard.

The vendor might not agree with that and change the implementation detail at their convenience. In absence of a prescriptive standard, a reference could be reasonably trusted, but relying on only sort-of-documented implementation details is a kind of tech debt.

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#129

Post by Kranar » Fri, Nov 17, 2023, 8:20 PM UTC

samus wrote: post #123

Kranar wrote: post #77

Earlier quoted context omitted.

That is not a specification. A specification is a prescriptive document of how a system is required to work whereas a reference is a descriptive document of how a system happens to currently work. Python has a reference, in fact it has many references that even contradict one another in subtle ways, but it does not have a specification.

Whether it is a specification or a reference doesn't really matter for this discussion. What matters is that it is the closest thing to a specification that exists for Python. Assuming anything beyond it ties the program to internals of a specific implementation. Internals for which the vendor might have different intentions about keeping stable or change than users expect.

It's absolutely pertinent to this discussion, this whole discussion is about whether the GIL is part of the semantics of a Python program or not. My position is that it is because the GIL is a part of the CPython reference implementation.

Others object saying that the Python reference document is what specifies its semantics, not the reference implementation.

My position is that both the CPython reference implementation and the reference documentation are valid sources that document Python's semantics and that they both can and should be used.

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#130

Post by samus » Fri, Nov 17, 2023, 8:32 PM UTC

nerdponx wrote: post #48

bheadmaster wrote: post #22

Earlier quoted context omitted.

> Just because everybody does something the wrong way doesn’t somehow make it magically correct. Depends on the definition of "correct". One way to define "correct" is "as defined by the standard". However, standard serves a purpose - to maintain interoperability between components, so the other, deeper, way to define "correct" is "interoperable with the ecosystem". The official standard can argue that relying on the…

Lest we forget that Python doesn't even have a language standard. The "standard" is a combination of PEPs, the Python docs, and the CPython implementation. Implementation details are language features, because implementation details are the standard. Python programs that rely on the GIL are not "wrong". Such programs are relying on clearly-documented features of the system that they use. Is it "wrong" to use GCC-spec…

GCC-specific features are explicitly documented in their manual, and they can and should be used if one is fine with one's program being tied to that compiler. Best example: the Linux kernel.

On the other hand, Linux kernel developers have also been burned by GCC changing what it does regarding undefined behavior (the infamous not-so-corner cases of C/C++). They expect GCC to be sort of like an assembler, which does the most straightforward thing in case of ambiguities. Instead, GCC is an optimizing compiler that is expected to deliver high-performance code for general-purpose programs. It treats undefined behavior as opportunities to enact optimizations. What it actually does is also usually documented, but also here different people have different expectations about what that implies.

The GIL is an implementation choice that was put in place by Python's developers for simplicity's sake. It is important to be aware of it because it has huge performance implications, but it is unwise to rely on it for semantics. Especially since it has not exactly been a secret that various parties would eventually like it to be removed. Anyways, the GIL has very little to do with Python's semantics. User code is racy with or without the GIL, which is actually the point of TA.