Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Changing "Guarantees" Given by Python's Global Interpreter Lock

stefan-marr.de

Post reply on HN
Print view View original

31–40 of 142 posts

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#31

Post by marcinzm »

darklion wrote: post #15
bheadmaster wrote: post #3

Earlier quoted context omitted.

True, but when a majority of ecosystem is relying on an implementation detail, that implementation detail becomes de-facto standard.

Just because everybody does something the wrong way doesn’t somehow make it magically correct. One study [1] in the US released in 2020 found almost 90% of people admitted to speeding, but I don’t think anyone would say that speeding is now approved by the authorities and consequence-free. [1] https://www.thezebra.com/resources/research/speeding-car-ins...

> One study [1] in the US released in 2020 found almost 90% of people admitted to speeding, but I don’t think anyone would say that speeding is now approved by the authorities and consequence-free.

I would say jaywalking in NYC is a much better example because breaking the law doesn't kill others except in rare cases. Jaywalking is illegal. It's also not enforced for all intents and purposes. Every attempt to enforce it has led to such loud public outcry that it was stopped.

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#32

Post by colanderman »

nicolaslem wrote: post #16
BiteCode_dev wrote: post #6

Earlier quoted context omitted.

The gil is a feature. Not using a feature that makes sense in our context is counter productive.

The GIL in an implementation detail of CPython, it's not part of Python the language.

It's been a while since I've followed Python closely but last I heard Python-the-language is de facto defined as "whatever CPython does". Has Python since grown a proper language specification?

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#33

Post by nerdponx »

dfox wrote: post #18
zzzeek wrote: post #9

these examples of "what one would assume to be atomic" did not seem useful to me, they looked like things that are obviously not threadsafe. a more interesting example is something like this: # setup l = [] # thread A l.extend([1, 2, 3]) # thread B l.extend([4, 5, 6]) is the resulting list always within the set of [1,2,3,4,5,6] or [4,5,6,1,2,3] ? or are the two sets of numbers randomly interleaved in the list? or if…

You can probably trigger the behavior of random interleaving even with GIL if you use something other than list or tuple as an argument to list.extend(). CPython special cases list and tuples and copies the contents directly (list_extend_fast() in listobject.c) while it uses iterator for other types (list_extend_iter()). Because the iterator can be pretty much arbitrary Python code it seems to be a bad idea to guaran…

All the more reason to wonder. What's going to be special-cased to become atomic and thread-safe, and what won't be?

A surprising amount of the CPython stdlib is just pure Python code.

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#34

Post by formerly_proven »

mikepurvis wrote: post #27
dfox wrote: post #2

Code that assumes that something is going to be atomic because of the GIL (or any other implementation detail) is simply broken. If you need something to be atomic you should be explicit about that and use mutex or something.

I dunno about that. There's a lot of Python out there with worker threads dumping their output into a shared "results" dict on the assumption that those insert operations are atomic.

Relying on implementation behavior, even if it’s very visible and hasn’t changed for a few decades, is morally wrong and reprehensible. Only the letter of the standard has any bearing on correct reality.

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#35

Post by formerly_proven »

colanderman wrote: post #32
nicolaslem wrote: post #16

Earlier quoted context omitted.

The GIL in an implementation detail of CPython, it's not part of Python the language.

It's been a while since I've followed Python closely but last I heard Python-the-language is de facto defined as "whatever CPython does". Has Python since grown a proper language specification?

No.

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#36

Post by irishsultan »

rcxdude wrote: post #8
coldcode wrote: post #4

Why does Python use an un-comparable version number scheme? Not being a Python programmer, comparing version 3.9 to 3.13 seemed bizarre until I caught on.

comparing version numbers as tuples and not as decimals is a pretty standard thing. Linux does it as well, for example. I am actually not sure of a project which does something different.

TeX and Metafont have version numbers that are approaching pi and e respectively, so the sensible way is to read these as decimals.

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#37

Post by mrkeen »

continuitylimit wrote: post #26

The brief blurb about how GIL came to be, in light of Python’s success as a language and a tool, makes me question my s/e belief system. Things like this are like when good things happen to bad people and bad things happen to good people. It makes you question the meaning of it all. Is there no great architect in the sky? Is there no software god after all, looking down, punishing sloppy engineers and granting blessi…

It's network effects, all the way down.

Watch people's commentary here when talking about the good and bad of various technologies.

There's no 'bad' tech, just tech with lots of users. Or the tech is good because you can hire for it. Because it has lots of users. Or it has a rich ecosystem, because it has lots of users.

Read the advice given by those who tell you to get to market first instead of polishing the tech.

We're the users of that tech which went to market unpolished and gathered all the users.

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#38

Post by nxpnsv »

throwaway468234 wrote: post #11
coldcode wrote: post #4

Why does Python use an un-comparable version number scheme? Not being a Python programmer, comparing version 3.9 to 3.13 seemed bizarre until I caught on.

Is there any versioning scheme that is comparable? Honest question.

TeX version number asymptotically approaches pi... each new version has another digit, this also makes versions comparable. Clearly this is the better way....

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#39

Post by yoyohello13 »

It seems like in every python discussion I hear people complain about the GIL.

I’m happy people are working on removing the GIL, but As a professional python dev for about 5 years now I have literally never had a problem where the GIL was a limiter. Although I just make web apps so maybe I’m not the target audience.

Re: The Changing "Guarantees" Given by Python's Global Interpreter Lock

#40

Post by jerf »

continuitylimit wrote: post #26

The brief blurb about how GIL came to be, in light of Python’s success as a language and a tool, makes me question my s/e belief system. Things like this are like when good things happen to bad people and bad things happen to good people. It makes you question the meaning of it all. Is there no great architect in the sky? Is there no software god after all, looking down, punishing sloppy engineers and granting blessi…

While there is some truth in what you are saying, you have a common misunderstanding of the situation. Part of the reason the GIL has proved so difficult to remove is that it is actually a good solution. In fact, there have been multiple largely successful attempts to remove it over time over the entire range of aggressiveness from CPython changes to writing an entire JIT stack (PyPy), but it has never gone in to CPython because it would either ruin all existing 3rd party libraries that used C (which is a lot of them), it would diminish performance for an already-slow language, or as in the case of PyPy, it isn't even a "patch" so much as a new project.

Especially when you consider this over the whole of Python's lifespan, which very, very firmly includes many years in which multicore was simply not a thing, followed by some years where it was a thing but it didn't work very well anyhow at the OS level so who cares what Python does with it.

It is not as if back when it was put it the choice was either to use a GIL or to correctly write a multithreaded interpreter and fix all the 3rd party libraries at the time for exactly the same cost. The latter option was orders of magnitude more expensive, and harder then than it is now, with better tooling and more collective developer experience. The choice of not using a GIL, rather than being some sort of nirvana that we could just be in if they hadn't chosen poorly 15 years ago, could well have killed the language. We don't really know. I do know that a programming language that just sort of breaks every so often when you use threads and there's absolutely nothing you can do about it from the Python level is not a very appealing proposition and it's hard to know how badly this could have hurt the language.

And Python of all the languages now has a well-justified fear of breaking everything and demanding that everyone upgrade.

So, to put this in a nutshell, if you believe the GIL is simply bad and should never have been an option, you have a very immature understanding of software engineering, especially in the light of being the leader of a very very large community who will be impacted by your decisions. It may not have been the only choice, but it was a good one, and regardless of what decision was made 15 years ago there would be some consequence to deal with now. No programming language community can be expected to get everything right in 2003 that the people of 2023 will want any more than we can expect any current programming language to be the perfect programming language of 2043 right this second.

Post reply on HN