Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Free-threaded CPython is ready to experiment with

labs.quansight.org

Post reply on HN
Print view View original

371–380 of 398 posts

Re: Free-threaded CPython is ready to experiment with

#371

Post by mabster »

skissane wrote: post #344
lyu07282 wrote: post #324

Earlier quoted context omitted.

There is a reason why it's "complicated" in threads, because doing it correctly just IS complicated, and the same reason applies to child processes, you just ignored that reason. That's one example of a footgun in using multiprocessing, people write broken code but they don't know that because it appears to work... until it doesn't (in production on friday night).

I don't agree. A big reason why abruptly terminating threads at an arbitrary point is risky is it can corrupt shared memory. If you aren't using shared memory in a multiprocess solution, that's not an issue. Another big reason is it can lead to resource leaks (e.g. thread gets terminated in a finally clause to close resources and hence the resource doesn't get closed). Again, that's less of an issue for processes, si…

That's why I've always liked Java's take on this. Throw an InterruptedException and the thread is considered terminated once that has dropped all the way through. You can also defer the exception for some time if it takes time to clean something up.

The only issue there is that sometimes library code will incorrectly defer the exception (i.e. suppress it) but otherwise it's pretty good.

Re: Free-threaded CPython is ready to experiment with

#372

Post by westurner »

westurner wrote: post #369

Will there be an effort to encourage devs to add support for free-threaded Python like for Python 3 [1] and for Wheels [2]? Is there a cibuildwheel / CI check for free-threaded Python support? Is there already a reason not to have Platform compatibility tags for free-threaded cpython support? https://packaging.python.org/en/latest/specifications/platfo... Is there a hame - a hashtaggable name - for this feature to he…

(2021) https://news.ycombinator.com/item?id=29005573#29009072 :

python-feedstock / recipe / meta.yml: https://github.com/conda-forge/python-feedstock/blob/master/...

pypy-meta-feedstock can be installed in the same env as python-feedstock; https://github.com/conda-forge/pypy-meta-feedstock/blob/main...

Re: Free-threaded CPython is ready to experiment with

#373

Post by dragonwriter »

Uptrenda wrote: post #341
wokwokwok wrote: post #238

Earlier quoted context omitted.

> there is just a tremendous amount of performance that can be automatically unlocked with almost no incremental effort for so many organizations and projects This just isn’t true. This does not improve single threaded performance (it’s worse) and concurrent programming is already available. This will make it less annoying to do concurrent processing. It also makes everything slower (arguable where that ends up, curr…

Why would it make single threaded performance slower? Sorry, but that's kind of ridiculous. You're just making shit up at this point.

Removing the GIL requires operations that are protected by it when it exists to be made thread safe in ways which they don't need to be with it, which has some overhead. Even having multiple options from which the situationally correct one (e.g., a lighter-weight one in guaranteed single-threaded or GIL active cases) can automatically be selected has some overhead. (Conceptually, you could have separate implementations that can be selected by the programmer with zero runtime overhead where not needed, but that also has more conceptual/developer overhead, so other than particular things where the runtime cost is found to really bite in practice, that's probably not going to be a common approach for core libraries that might need to be called either way. There's no free lunch here.

Re: Free-threaded CPython is ready to experiment with

#374

Post by fuzztester »

jacob019 wrote: post #171
fuzztester wrote: post #163

Earlier quoted context omitted.

[flagged]

It's very unpopular to mention Perl, but I did many cool things with it back in the day, and it still holds a special place for me. Perl taught me the power of regex--it's really first class in Perl. I still have some Perl code in production today. But to be fair, it is really easy to write spaghetti in Perl if you don't know what you're doing.

> But to be fair, it is really easy to write spaghetti in Perl if you don't know what you're doing.

bro, you really need to educate yourself some more, if you say things like that.

baloney!

first of all, perl is already spaghetti. I mean, it has all those curlicues, aka sigils. perl devs love it that way.

second of all, that point about being able to write spaghetti is not unique to perl.

many people do it in many languages.

in fact it is common here on hn to see the statement "you can write fortran in any language".

Re: Free-threaded CPython is ready to experiment with

#375

Post by fuzztester »

porksoda wrote: post #266
fuzztester wrote: post #221

Earlier quoted context omitted.

[flagged]

Your comments feel like you want a reddit style battle of wits. You throw words like ignorant around rather freely. I downvote these comments because they lower the tone of the discussion. We're not here to talk about each other or feel smarter than others.. Well I'm not.

I don't give a flying fuck about your feelings or your opinions about my comments, just as you should not about mine. i am a free man. you should try to be one too. we can talk to each other, but that does not mean that either of us has to believe or be convinced by what the other person says. is this not fuckingly blindlingly obvious to you?

if not, you have a serious perception problem.

yes, I do throw around words, just like anyone else, but not freely, instead, I do that after at least some thought, which I do not see happening in the people whom I replied to, nor in your comment above.

Re: Free-threaded CPython is ready to experiment with

#376

Post by fuzztester »

porksoda wrote: post #266
fuzztester wrote: post #221

Earlier quoted context omitted.

[flagged]

Your comments feel like you want a reddit style battle of wits. You throw words like ignorant around rather freely. I downvote these comments because they lower the tone of the discussion. We're not here to talk about each other or feel smarter than others.. Well I'm not.

[flagged]

Re: Free-threaded CPython is ready to experiment with

#377

Post by arp242 »

simonw wrote: post #285
arp242 wrote: post #239

Earlier quoted context omitted.

Right so; I'll try that next time. Thanks. I just go by the very prominent "pip install X" on every pypi page (as well as "pip install .." in many READMEs).

Yeah, totally understand that - pipx is still pretty poorly known by people who are active in Python development! A few of my READMEs start like this: https://github.com/simonw/paginate-json?tab=readme-ov-file#i... ## Installation pip install paginate-json Or use pipx (link to pipx site) pipx install paginate-json But I checked and actually most them still don't even mention it. I'll be fixing that in the future.

Out of the 3 things I want to install 2 don't work. Both of these seem bugs in pipx so I reported one, but the feedback was borderline hostile and it ended up being closed with "unclear what you want". I'm not even going to bother reporting the other bug.

So whatever the goals are, it doesn't really work. And in general pipx does not strike me as a serious project.

Re: Free-threaded CPython is ready to experiment with

#378

Post by wokwokwok »

Uptrenda wrote: post #341
wokwokwok wrote: post #238

Earlier quoted context omitted.

> there is just a tremendous amount of performance that can be automatically unlocked with almost no incremental effort for so many organizations and projects This just isn’t true. This does not improve single threaded performance (it’s worse) and concurrent programming is already available. This will make it less annoying to do concurrent processing. It also makes everything slower (arguable where that ends up, curr…

Why would it make single threaded performance slower? Sorry, but that's kind of ridiculous. You're just making shit up at this point.

https://news.ycombinator.com/item?id=40949564

^ read. The OP responds in the thread.

tldr, literally what I said:

> It also makes everything slower (arguable where that ends up, currently significantly slower) overall.

longer version:

If there was no reason for it to be slower, it would not be slower.

...but, implementing this stuff is hard.

Doing a zero cost implementation is really hard.

It is slower.

Where it ends up eventually is still a 'hm... we'll see'.

To be fair, they didn't lead the article here with:

> Right now there is a significant single-threaded performance cost. Somewhere from 30-50%.

They should have, because now people have a misguided idea of what this wip release is... and that's not ideal; because if you install it, you'll find its slow as balls; and that's not really the message they were trying to put out with this release. This release was about being technically correct.

...but, it is slow as balls right now, and I'm not making that shit up. Try it yourself.

/shrug

Re: Free-threaded CPython is ready to experiment with

#379

Post by skissane »

lyu07282 wrote: post #354
skissane wrote: post #353

Earlier quoted context omitted.

> This is not a matter of opinion. Always clean up after yourself, the kernel doesn't know shit about your application or what state it's in, you can not rely on it to cleanly terminate your process. If you have an open file or network connection, the kernel is guaranteed to close it for you when the process is killed (assuming it hasn't passed the fd/socket to a subprocess, etc). That's not a matter of opinion. Yes,…

Oh well I see, you will learn the hard way then (:

You have no idea what I'm actually doing, yet you are convinced something bad is bound to happen, although you can't say what exactly that bad thing will be.

That's not useful feedback.

Re: Free-threaded CPython is ready to experiment with

#380

Post by robertlagrant »

throwaway81523 wrote: post #325
pansa2 wrote: post #248

Earlier quoted context omitted.

Yes, this is similar to how Go works. IIRC the same approach was available in Python as a library, “greenlet”, but Python’s core developers rejected it in favour of `async`/`await`.

The Python community seems to have a virulent hatred of threads. I don't understand the reason. Yes there are hazards but you can code in a style that avoids them. With something like BEAM you can even enforce the style. Async/await of course introduce their own hazards.

There's no hatred. There are just lots of libraries that can't be multithreaded, due to historical reasons. This is being worked on right now[0], though.

[0] https://peps.python.org/pep-0703

Post reply on HN