Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The first year of free-threaded Python

labs.quansight.org

Post reply on HN
Print view View original

111–120 of 302 posts

Re: The first year of free-threaded Python

#111

Post by mtzaldo »

vlovich123 wrote: post #106
pjmlp wrote: post #40

On the other news, Microsoft dumped the whole faster Python team, apparently the 2025 earnings weren't enough to keep the team around. https://www.linkedin.com/posts/mdboom_its-been-a-tough-coupl... Lets see whatever performance improvements still land on CPython, unless other company sponsors the work. I guess Facebook (no need to correct me on the name) is still sponsoring part of it.

That’s unfortunate but I called it when people were claiming that Microsoft had committed to this effort for the long term.

Could we do a crowdfunding campaign so we can keep paying them? The whole world is/will benefit from their work.

Re: The first year of free-threaded Python

#112

Post by chubot »

sgarland wrote: post #48

> Instead, many reach for multiprocessing, but spawning processes is expensive Agreed. > and communicating across processes often requires making expensive copies of data SharedMemory [0] exists. Never understood why this isn’t used more frequently. There’s even a ShareableList which does exactly what it sounds like, and is awesome. [0]: https://docs.python.org/3/library/multiprocessing.shared_mem...

Spawning processes generally takes much less than 1 ms on Unix

Spawning a PYTHON interpreter process might take 30 ms to 300 ms before you get to main(), depending on the number of imports

It's 1 to 2 orders of magnitude difference, so it's worth being precise

This is a fallacy with say CGI. A CGI in C, Rust, or Go works perfectly well.

e.g. sqlite.org runs with a process PER REQUEST - https://news.ycombinator.com/item?id=3036124

Re: The first year of free-threaded Python

#113

Post by hello_computer »

Opting to enable low-level parallelism for user code in an imperative, dynamically typed scripting language seems like regression. It’s less bad for LISP because of the pure-functional nature. It’s less bad for BEAM languages & Clojure due to immutability. It is less bad for C/C++/Rust because you have a stronger type system—allowing for deeper static analysis. For Python, this is “high priests of a low cult” shitting things up for corporate agendas and/or street cred.

Re: The first year of free-threaded Python

#114

Post by almostgotcaught »

AlexanderDhoore wrote: post #2

Am I the only one who sort of fears the day when Python loses the GIL? I don't think Python developers know what they’re asking for. I don't really trust complex multithreaded code in any language. Python, with its dynamic nature, I trust least of all.

Do you understand what you're implying?

"Python programmers are so incompetent that Python succeeds as a language only because it lacks features they wouldn't know to use"

Even if it's circumstantially true, doesn't mean it's the right guiding principle for the design of the language.

Re: The first year of free-threaded Python

#116

Post by pphysch »

celeritascelery wrote: post #98
rfoo wrote: post #26

Earlier quoted context omitted.

> Does free-threaded Python provide the same guarantees Mostly. Some of the "can be pre-empted on the boundary between any two bytecode instructions" bugs are really hard to hit without free-threading, though. And without free-threading people don't use as much threading stuff. So by nature it exposes more bugs. Now, my rants: > have any other effects on multi-threaded Python code It stops people from using multi-pro…

> That's the trade-off. Personally I think a single digit percentage slow-down of single-threaded code worth it. Maybe. I would expect that 99% of python code going forward will still be single threaded. You just don’t need that extra complexity for most code. So I would expect that python code as a whole will have worse performance, even though a handful of applications will get faster.

But the bar to parallelizing code gets much lower, in theory. Your serial code got 5% slower but has a direct path to being 50% faster.

And if there's a good free-threaded HTTP server implementation, the RPS of "Python code as a whole" could increase dramatically.

Re: The first year of free-threaded Python

#117

Post by Sharlin »

chubot wrote: post #112
sgarland wrote: post #48

> Instead, many reach for multiprocessing, but spawning processes is expensive Agreed. > and communicating across processes often requires making expensive copies of data SharedMemory [0] exists. Never understood why this isn’t used more frequently. There’s even a ShareableList which does exactly what it sounds like, and is awesome. [0]: https://docs.python.org/3/library/multiprocessing.shared_mem...

Spawning processes generally takes much less than 1 ms on Unix Spawning a PYTHON interpreter process might take 30 ms to 300 ms before you get to main(), depending on the number of imports It's 1 to 2 orders of magnitude difference, so it's worth being precise This is a fallacy with say CGI. A CGI in C, Rust, or Go works perfectly well. e.g. sqlite.org runs with a process PER REQUEST - https://news.ycombinator.com/it…

Unix is not the only platform though (and is process creation fast on all Unices or just Linux?) The point about interpreter init overhead is, of course, apt.

Re: The first year of free-threaded Python

#118

Post by indymike »

delusional wrote: post #71
bayindirh wrote: post #42

Earlier quoted context omitted.

Software rots, software tools evolve. When Intel released performance primitives libraries which required recompilation to analyze multi-threaded libraries, we were amazed. Now, these tools are built into processors as performance counters and we have way more advanced tools to analyze how systems behave. Older code will break, but they break all the time. A language changes how something behaves in a new revision, s…

> Software rots No it does not. I hate that analogy so much because it leads to such bad behavior. Software is a digital artifact that can does not degrade. With the right attitude, you'd be able to execute the same binary on new machines for as long as you desired. That is not true of organic matter that actually rots. The only reason we need to change software is that we trade that off against something else. Instr…

>> Software rots > No it does not.

I'm thankful that it does, or I would have been out of work long ago. It's not that the files change (literal rot), it is that hardware, OSes, libraries, and everything else changes. I'm also thankful that we have not stopped innovating on all of the things the software I write depends on. You know, another thing changes - what we are using the software for. The accounting software I wrote in the late 80s... would produce financial reports that were what was expected then, but would not meet modern GAAP requirements.

Re: The first year of free-threaded Python

#119

Post by kstrauser »

oblio wrote: post #100
kstrauser wrote: post #73

Earlier quoted context omitted.

cough Bullshit cough * VSCode got popular and they started preventing forks from installing its extensions. * They extended the Free Source pyright language server into the proprietary pylance. They don’t even sell it. It’s just there to make the FOSS version less useful. * They bought GitHub and started rate limiting it to unlogged in visitors. Every time Microsoft touches a thing, they end up locking it down. They…

I actually hate this trope more because of what is says about the poster. Which I guess would, that they're someone wearing horse blinders. There's a part of me that wants to scream at them: "Look around you!!! It's not 1999 anymore!!! These days we have Google, Amazon, Apple, Facebook, etc, which are just as bad if not worse!!! Cut it out with the 20+ year old bad jokes!!!" Yes, Microsoft is bad. The reason Micr$oft…

That’s only reasonable if you believe you can only distrust one company at a time. I distrust every one you mentioned there, for different reasons, in different ways. I don’t think that Apple is trying to exclusively own the field of programming tools to their own profit, nor do I think that Facebook is. I don’t think Apple is trying to own all data about every human. I don’t think Microsoft is trying to force all vendors to sell through their app store.

But the thing is that Microsoft hasn’t seemed to fundamentally change since 1999. They appear kinder and friendlier but they keep running the same EEE playbook everywhere they can. Lots of us give them a free pass because they let us run a nifty free-for-now programming editor. That doesn’t change the leopard’s spots, though.

Re: The first year of free-threaded Python

#120

Post by jerrygenser »

stusmall wrote: post #88
rich_sasha wrote: post #46

Earlier quoted context omitted.

Ah that's very, very sad. I guess they have embraced and extended, there's only one thing left to do.

That shows a misunderstanding of what EEE was. This team was sending changes upstream which is the exact opposite of "extend" step of the strategy. The idea of "extend" was to add propriety extensions on top of an open standard/project locking customers into the MSFT implementation.

Ok so a better example of what you describe might be vscode.
Post reply on HN