Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The first year of free-threaded Python

labs.quansight.org

Post reply on HN
Print view View original

121–130 of 302 posts

Re: The first year of free-threaded Python

#121

Post by morkalork »

pjmlp wrote: post #40

On the other news, Microsoft dumped the whole faster Python team, apparently the 2025 earnings weren't enough to keep the team around. https://www.linkedin.com/posts/mdboom_its-been-a-tough-coupl... Lets see whatever performance improvements still land on CPython, unless other company sponsors the work. I guess Facebook (no need to correct me on the name) is still sponsoring part of it.

Didn't Google lay off their entire Python development team in the last year as well? I wonder if there is some impetus behind both.

Re: The first year of free-threaded Python

#122

Post by rowanG077 »

ynik wrote: post #95
rowanG077 wrote: post #25

Earlier quoted context omitted.

Afaik the only guarantee there is, is that a bytecode instruction is atomic. Built in data structures are mostly safe I think on a per operation level. But combining them is not. I think by default every few millisecond the interpreter checks for other threads to run even if there is no IO or async actions. See `sys.getswitchinterval()`

Bytecode instructions have never been atomic in Python's past. It was always possible for the GIL to be temporarily released, then reacquired, in the middle of operations implemented in C. This happens because C code is often manipulating the reference count of Python objects, e.g. via the `Py_DECREF` macro. But when a reference count reaches 0, this might run a `__del__` function implemented in Python, which means t…

Well I didn't think of this myself. It's literally what the python official doc says:

> A global interpreter lock (GIL) is used internally to ensure that only one thread runs in the Python VM at a time. In general, Python offers to switch among threads only between bytecode instructions; how frequently it switches can be set via sys.setswitchinterval(). Each bytecode instruction and therefore all the C implementation code reached from each instruction is therefore atomic from the point of view of a Python program.

https://docs.python.org/3/faq/library.html#what-kinds-of-glo...

If this is not the case please let the official python team know their documentation is wrong. It indeed does state that if Py_DECREF is invoked the bets are off. But a ton of operations never do that.

Re: The first year of free-threaded Python

#123

Post by nothrabannosir »

jerrygenser wrote: post #120
stusmall wrote: post #88

Earlier quoted context omitted.

That shows a misunderstanding of what EEE was. This team was sending changes upstream which is the exact opposite of "extend" step of the strategy. The idea of "extend" was to add propriety extensions on top of an open standard/project locking customers into the MSFT implementation.

Ok so a better example of what you describe might be vscode.

What existing open standard did vscode Embrace? I thought Microsoft created v0 themselves.

A classic example is ActiveX.

Re: The first year of free-threaded Python

#124

Post by kstrauser »

YouWhy wrote: post #65

Hey, I've been developing professionally with Python for 20 years, so wanted to weigh in: Decent threading is awesome news, but it only affects a small minority of use cases. Threads are only strictly necessary when it's prohibitive to message pass. The Python ecosystem these days includes a playbook solution for literally any such case. Considering the multiple major pitfalls of threads (i.e., locking), they are lik…

I haven’t been using it that much longer than you, and I agree with most of what you’re saying, but I’d characterize it differently.

Python has a lot of solid workarounds for avoid threading because until now Python threading has absolutely sucked. I had naively tried to use it to make a CPU-bound workload twice as fast and soon realized the implications of the GIL, so I threw all that code away and made it multiprocessing instead. That sucked in its own way because I had to serialize lots of large data structures to pass around, so 2x the cores got me about 1.5x the speed and a warmer server room.

I would love to have good threading support in Python. It’s not always the right solution, but there are a lot of circumstances where it’d be absolutely peachy, and today we’re faking our way around its absence with whole playbooks of alternative approaches to avoid the elephant in the room.

But yes, use async when it makes sense. It’s a thing of beauty. (Yes, Glyph, we hear the “I told you so!” You were right.)

Re: The first year of free-threaded Python

#125

Post by biorach »

kstrauser wrote: post #73
biorach wrote: post #59

Earlier quoted context omitted.

At this stage the cliched and clueless comments about embrace/extend/extinguish are tiresome and inevitable whenever Microsoft is mentioned. A few decades ago MS did indeed have a playbook which they used to undermine open standards. Laying off some members of the Python team bears no resemblence whatsoever to that. At worst it will delay the improvement of free-threaded Python. That's all. Your comment is lazy and u…

cough Bullshit cough * VSCode got popular and they started preventing forks from installing its extensions. * They extended the Free Source pyright language server into the proprietary pylance. They don’t even sell it. It’s just there to make the FOSS version less useful. * They bought GitHub and started rate limiting it to unlogged in visitors. Every time Microsoft touches a thing, they end up locking it down. They…

None of those were independent projects or open standards. VScode and pyright are both MS projects from the get-go.

Sabotaging forks is scummy, but the forks were extending MS functionality, not the other way around.

GitHub was a private company before it was bought by MS. Rate limiting is.... not great, but certainly not an extinguish play.

EEE refers to the subversion of open standards or independent free software projects. It does not apply to any of the above.

MS are still scummy but at least attack them on their own demerits, and don't parrot some schtick from decades ago.

Re: The first year of free-threaded Python

#126

Post by rocqua »

rfoo wrote: post #26
pansa2 wrote: post #19

Does removal of the GIL have any other effects on multi-threaded Python code (other than allowing it to run in parallel)? My understanding is that the GIL has lasted this long not because multi-threaded Python depends on it, but because removing it: - Complicates the implementation of the interpreter - Complicates C extensions, and - Causes single-threaded code to run slower Multi-threaded Python code already has to…

> Does free-threaded Python provide the same guarantees Mostly. Some of the "can be pre-empted on the boundary between any two bytecode instructions" bugs are really hard to hit without free-threading, though. And without free-threading people don't use as much threading stuff. So by nature it exposes more bugs. Now, my rants: > have any other effects on multi-threaded Python code It stops people from using multi-pro…

Note that there is an entire order of magnitude range for a 'single digit'.

A 1% slowdown seems totally fine. A 9% slowdown is pretty bad.

Re: The first year of free-threaded Python

#127

Post by biorach »

nothrabannosir wrote: post #123
jerrygenser wrote: post #120

Earlier quoted context omitted.

Ok so a better example of what you describe might be vscode.

What existing open standard did vscode Embrace? I thought Microsoft created v0 themselves. A classic example is ActiveX.

> A classic example is ActiveX.

Nah, even that was based on earlier MS technologies - OLE and COM

A good starter list of EEE plays is on the wikipedia page: https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...

Re: The first year of free-threaded Python

#128

Post by mixmastamyk »

kstrauser wrote: post #119
oblio wrote: post #100

Earlier quoted context omitted.

I actually hate this trope more because of what is says about the poster. Which I guess would, that they're someone wearing horse blinders. There's a part of me that wants to scream at them: "Look around you!!! It's not 1999 anymore!!! These days we have Google, Amazon, Apple, Facebook, etc, which are just as bad if not worse!!! Cut it out with the 20+ year old bad jokes!!!" Yes, Microsoft is bad. The reason Micr$oft…

That’s only reasonable if you believe you can only distrust one company at a time. I distrust every one you mentioned there, for different reasons, in different ways. I don’t think that Apple is trying to exclusively own the field of programming tools to their own profit, nor do I think that Facebook is. I don’t think Apple is trying to own all data about every human. I don’t think Microsoft is trying to force all ve…

All these posts and no one mentioned their numerous, recent, abusive deeds around Windows or negligent security posture, all the while having captured Uncle Sam and other governments.

MS has continued to metastasize and is in some ways worse than the old days, even if they’ve finally accepted the utility of open source as a loss leader.

They have the only BigTech products I’ve been forced to use if I want to eat.

Re: The first year of free-threaded Python

#129

Post by igouy »

eblume wrote: post #80
delusional wrote: post #71

Earlier quoted context omitted.

> Software rots No it does not. I hate that analogy so much because it leads to such bad behavior. Software is a digital artifact that can does not degrade. With the right attitude, you'd be able to execute the same binary on new machines for as long as you desired. That is not true of organic matter that actually rots. The only reason we need to change software is that we trade that off against something else. Instr…

Software is written with a context, and the context degrades. It must be renewed. It rots, sorry.

You said it's the context that rots.
Post reply on HN