Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Changing "Guarantees" Given by Python's Global Interpreter Lock

stefan-marr.de

Post reply on HN
Print view View original

61–70 of 142 posts

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#61

Post by quietbritishjim »

svara wrote: post #57
yoyohello13 wrote: post #39

It seems like in every python discussion I hear people complain about the GIL. I’m happy people are working on removing the GIL, but As a professional python dev for about 5 years now I have literally never had a problem where the GIL was a limiter. Although I just make web apps so maybe I’m not the target audience.

It's just about what kind of code you write. If you write a lot of code that parallelizes over data you will hurt all the time because of the GIL. If you have worker processes that do something on the CPU, and the results need to be collected and processed further in some other process, you now need to pickle the data to copy it around. That can get really slow. I understand a lot of python users don't do that kind o…

> If you write a lot of code that parallelizes over data [and that parallelization can't happen by vectorizing your operations in a C extension module that releases the GIL, like numpy] you will hurt all the time because of the GIL.

I've added some text for clarity

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#62

Post by quietbritishjim »

HDThoreaun wrote: post #47
colesbury wrote: post #23

Earlier quoted context omitted.

You will always get either [1,2,3,4,5,6] or [4,5,6,1,2,3] in the upcoming `--disable-gil` builds of CPython 3.13 and the nogil forks. Most operations on mutable collections hold a per-object lock. Part of the integration work will be to better document the thread-safety guarantees, but there is still a lot of work to do before we get there.

Every mutable collection needs to lock before write? Sounds slow af

Locks are generally very cheap if they're not contended. Of course it's all relative though!

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#64

Post by coldtea »

coldcode wrote: post #4

Why does Python use an un-comparable version number scheme? Not being a Python programmer, comparing version 3.9 to 3.13 seemed bizarre until I caught on.

You mean uncomparable using numeric or string sorting on those strings?

Because otherwise it's perfectly comparable - and quite common.

Most FOSS uses a variant of .. (here just major (3) + minor (13), minor meaning "release within the same major Python version").

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#65

Post by [deleted] »

formerly_proven wrote: post #34
mikepurvis wrote: post #27

Earlier quoted context omitted.

I dunno about that. There's a lot of Python out there with worker threads dumping their output into a shared "results" dict on the assumption that those insert operations are atomic.

Relying on implementation behavior, even if it’s very visible and hasn’t changed for a few decades, is morally wrong and reprehensible. Only the letter of the standard has any bearing on correct reality.

[deleted]

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#66

Post by pdonis »

dragonwriter wrote: post #58
pdonis wrote: post #54

Earlier quoted context omitted.

> Python doesn't have a full specification This doesn't count? https://docs.python.org/3/reference/index.html

I would say that that is intended as documentation for users (includong implementors of tools targeting the language), not a specification for implementors of Python, but I would agree that it is largely usable in either role; my point in the GP was that something distinct called “the Python Language Specification” doesn't exist, but that a (not necessarily complete, from a language implementors perspective) specific…

> not a specification for implementors of Python

I don't see why not. The "Introduction" section specifically mentions different implementations and distinguishes implementation details, which can vary by implementation, from the language reference itself, which defines what every implementation has to meet to be considered an implementation of the Python language.

> my point in the GP was that something distinct called “the Python Language Specification” doesn't exist

And that's the point I'm disputing; AFAIK the language reference I linked to is that something distinct, even if it isn't called a "Language Specification" but instead a "Language Reference". Either way it defines what the Python language is.

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#67

Post by fbdab103 »

HDThoreaun wrote: post #47
colesbury wrote: post #23

Earlier quoted context omitted.

You will always get either [1,2,3,4,5,6] or [4,5,6,1,2,3] in the upcoming `--disable-gil` builds of CPython 3.13 and the nogil forks. Most operations on mutable collections hold a per-object lock. Part of the integration work will be to better document the thread-safety guarantees, but there is still a lot of work to do before we get there.

Every mutable collection needs to lock before write? Sounds slow af

Last I looked, the nogil implementation was some 5-10% slower than the current, owing to all of the extra locking.

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#68

Post by coldtea »

jjoonathan wrote: post #12
coldcode wrote: post #4

Why does Python use an un-comparable version number scheme? Not being a Python programmer, comparing version 3.9 to 3.13 seemed bizarre until I caught on.

I won't argue with bizzare, but it's common in version notation. If it wasn't already common, it would probably become common shortly after the first time a big respectable company hit the "oops what comes after 9" problem and decided on dot-separated-integers rather than significant digits :)

Is it that bizarre?

It's basically semantic versioning, that is a hierarchical split based on levels of change (major = new release with possibly big breaking changes, minor = some incremental update version within the same release) and so on.

Who ever thought version numbers are decimals and why? The "." appears as a separator on all kinds of strings in software (filenames, domains, and IPs probably the most common ones).

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#69

Post by pdonis »

squeaky-clean wrote: post #56
pdonis wrote: post #54

Earlier quoted context omitted.

> Python doesn't have a full specification This doesn't count? https://docs.python.org/3/reference/index.html

The introduction section states that it's not a complete/exact specification. If you're using a Python implementation and have a question, this should answer it. If you're writing a Python implementation and have a question, this may not answer it. > Consequently, if you were coming from Mars and tried to re-implement Python from this document alone, you might have to guess things and in fact you would probably end u…

> If you're writing a Python implementation and have a question, this may not answer it.

Depends on the question. If your question is, say, "how should I implement the built-in types", yes, the language reference won't answer that question. But if your question is "what does my implementation have to do to count as an implementation of the Python language", then yes, the language reference does answer that question--since the language reference is what defines the Python language.

To me that is a "specification" of the Python language. It's not a specification of the implementation of the language, but why should that be required for something to be considered a language specification? The whole point is to specify what is required to define the language without specifying every detail of the implementation.

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#70

Post by Kranar »

formerly_proven wrote: post #34
mikepurvis wrote: post #27

Earlier quoted context omitted.

I dunno about that. There's a lot of Python out there with worker threads dumping their output into a shared "results" dict on the assumption that those insert operations are atomic.

Relying on implementation behavior, even if it’s very visible and hasn’t changed for a few decades, is morally wrong and reprehensible. Only the letter of the standard has any bearing on correct reality.

Python's semantics do not have a standards document or specification. It's based on its reference implementation (CPython) and different variations of Python even have different semantics (Jython, IronPython, PyPy).

The library is documented, and the syntax is also documented, but the semantics themselves are not.

Post reply on HN