Live data from Hacker News

The first year of free-threaded Python

labs.quansight.org

91–100 of 302 posts

Re: The first year of free-threaded Python

#91
post #5

Earlier quoted context omitted.

how does the the language being dynamic negatively affect the complexity of multithreading?

When the language is dynamic there is less rigor. Statically checked code is more likely to be correct. When you add threads to "fast and loose" code things get really bad.

Unless your claim is that the same error can happen more times per minute because threading can execute more code in the same timespan, this makes no sense.

Re: The first year of free-threaded Python

#92
post #68

Earlier quoted context omitted.

> Well, you sure managed to avoid that by setting up camp on that hill. Kudos on so much time saved. Why are you picking a fight about this?

I think I'm taking it personally because I had previously changed my name and had people repeatedly call me by my old name just to annoy/hurt me. Obviously I know that companies aren't people and don't have feelings, but I can't understand why you would intentionally avoid using their chosen name, even when it's more effort to you.

I wouldn’t do that to a person. I’m not worried about hurting Twitter’s feelings, though.

Re: The first year of free-threaded Python

#95
post #22

Earlier quoted context omitted.

With the GIL, multithreaded Python gives concurrent I/O without worrying about data structure concurrency (unless you do I/O in the middle of it) - it's a lot like async in this way - data structure manipulation is atomic between "await" expressions (except in the "await" is implicit and you might have written one without realizing in which case you have a bug). Meanwhile you still get to use threads to handle severa…

Afaik the only guarantee there is, is that a bytecode instruction is atomic. Built in data structures are mostly safe I think on a per operation level. But combining them is not. I think by default every few millisecond the interpreter checks for other threads to run even if there is no IO or async actions. See `sys.getswitchinterval()`

Bytecode instructions have never been atomic in Python's past. It was always possible for the GIL to be temporarily released, then reacquired, in the middle of operations implemented in C. This happens because C code is often manipulating the reference count of Python objects, e.g. via the `Py_DECREF` macro. But when a reference count reaches 0, this might run a `__del__` function implemented in Python, which means the "between bytecode instructions" thread switch can happen inside that reference-counting-operation. That's a lot of possible places!

Even more fun: allocating memory could trigger Python's garbage collector which would also run `__del_-` functions. So every allocation was also a possible (but rare) thread switch.

The GIL was only ever intended to protect Python's internal state (esp. the reference counts themselves); any extension modules assuming that their own state would also be protected were likely already mistaken.

Re: The first year of free-threaded Python

#96
post #86
post #58

Earlier quoted context omitted.

Well, I think you can manipulate a dict from two different threads in Python, today, without any risk of segfaults.

You can do so in free-threaded Python too, right? The dict is still protected by a lock, but one that’s much more fine-grained than the GIL.

Sounds good, yes.

Re: The first year of free-threaded Python

#98
post #26
post #19

Does removal of the GIL have any other effects on multi-threaded Python code (other than allowing it to run in parallel)? My understanding is that the GIL has lasted this long not because multi-threaded Python depends on it, but because removing it: - Complicates the implementation of the interpreter - Complicates C extensions, and - Causes single-threaded code to run slower Multi-threaded Python code already has to…

> Does free-threaded Python provide the same guarantees Mostly. Some of the "can be pre-empted on the boundary between any two bytecode instructions" bugs are really hard to hit without free-threading, though. And without free-threading people don't use as much threading stuff. So by nature it exposes more bugs. Now, my rants: > have any other effects on multi-threaded Python code It stops people from using multi-pro…

> That's the trade-off. Personally I think a single digit percentage slow-down of single-threaded code worth it.

Maybe. I would expect that 99% of python code going forward will still be single threaded. You just don’t need that extra complexity for most code. So I would expect that python code as a whole will have worse performance, even though a handful of applications will get faster.

Re: The first year of free-threaded Python

#99
post #5

Am I the only one who sort of fears the day when Python loses the GIL? I don't think Python developers know what they’re asking for. I don't really trust complex multithreaded code in any language. Python, with its dynamic nature, I trust least of all.

how does the the language being dynamic negatively affect the complexity of multithreading?

I have a hypothesis that being dynamic has no particular effect on the complexity of multithreading. I think the apparent effect is a combination of two things: 1. All our dynamic scripting languages in modern use date from the 1990s before this degree of threading was a concern for the languages and 2. It is really hard to retrofit code written for not being threaded to work in a threaded context, and the "deeper" the code in the system the harder it is. Something like CPython is about as "deep" as you can go, so it's really, really hard.

I think if someone set out to write a new dynamic scripting language today, from scratch, that multithreading it would not pose any particular challenge. Beyond that fact that it's naturally a difficult problem, I mean, but nothing special compared to the many other languages that have implemented threading. It's all about all that code from before the threading era that's the problem, not the threading itself. And Python has a loooot of that code.

Re: The first year of free-threaded Python

#100
post #59

Earlier quoted context omitted.

At this stage the cliched and clueless comments about embrace/extend/extinguish are tiresome and inevitable whenever Microsoft is mentioned. A few decades ago MS did indeed have a playbook which they used to undermine open standards. Laying off some members of the Python team bears no resemblence whatsoever to that. At worst it will delay the improvement of free-threaded Python. That's all. Your comment is lazy and u…

cough Bullshit cough * VSCode got popular and they started preventing forks from installing its extensions. * They extended the Free Source pyright language server into the proprietary pylance. They don’t even sell it. It’s just there to make the FOSS version less useful. * They bought GitHub and started rate limiting it to unlogged in visitors. Every time Microsoft touches a thing, they end up locking it down. They…

I actually hate this trope more because of what is says about the poster. Which I guess would, that they're someone wearing horse blinders.

There's a part of me that wants to scream at them:

"Look around you!!! It's not 1999 anymore!!! These days we have Google, Amazon, Apple, Facebook, etc, which are just as bad if not worse!!! Cut it out with the 20+ year old bad jokes!!!"

Yes, Microsoft is bad. The reason Micr$oft was the enemy back in the day is because they... won. They were bigger than anyone else in the fields that mattered (except for server-side, where they almost one). Now they're just 1 in a gang of evils. There's nothing special about them anymore. I'm more scared of Apple and Google.

Post reply on HN