Free-threaded CPython is ready to experiment with • Hacker News Board

Re: Free-threaded CPython is ready to experiment with

#231

Post by pizza234 » Sat, Jul 13, 2024, 6:30 AM UTC

eigenvalue wrote: post #12

Really excited for this. Once some more time goes by and the most important python libraries update to support no GIL, there is just a tremendous amount of performance that can be automatically unlocked with almost no incremental effort for so many organizations and projects. It's also a good opportunity for new and more actively maintained projects to take market share from older and more established libraries if th…

> using simple threads instead of dealing with the massive overhead and complexity and bugs of using something like multiprocessing.

Depending on the domain, the reality can be the reverse.

Multiprocessing in the web serving domain, as in "spawning separate processes", is actually simpler and less bug-prone, because there is considerably less resource sharing. The considerably higher difficulty of writing, testing and debugging parallel code is evident to anybody who's worked on it.

As for the overhead, this again depends on the domain. It's hard to quantify, but generalizing to "massive" is not accurate, especially for app servers with COW support.

Re: Free-threaded CPython is ready to experiment with

#232

Post by KeplerBoy » Sat, Jul 13, 2024, 6:40 AM UTC

agumonkey wrote: post #47

nine_k wrote: post #6

Python 3 progress so far: [x] Async. [x] Optional static typing. [x] Threading. [ ] JIT. [ ] Efficient dependency management.

I'm eager to see what a simple JIT can bring to computing energy savings on python apps.

I'd wager the energy savings could put multiple power plants out of service.

I regularly encounter python code which takes minutes to execute but runs in less than a second when replacing key parts with compiled code.

Re: Free-threaded CPython is ready to experiment with

#233

Post by KeplerBoy » Sat, Jul 13, 2024, 6:43 AM UTC

veber-alex wrote: post #207

jmward01 wrote: post #105

I know, I know, 'not every story needs to be about ML' but.... I can only imagine how unlocking the GIL will change the nature of ML training and inference. There is so much waste and complexity in passing memory around and coordinating processes. I know that libraries have made it (somewhat) easier and more efficient but I can't wait to see what can be done with things like pytorch when optimized for this.

huh? Any python library that cares about performance is written in C/C++/Rust/Fortran and only provides a python interface. ML will have 0 benefit from this.

Of course ML will benefit from it. Soon you will be able to run your dataloaders/data preprocessing in different threads which will not starve your GPUs of data.

Re: Free-threaded CPython is ready to experiment with

#234

Post by throwaway81523 » Sat, Jul 13, 2024, 6:51 AM UTC

robertlagrant wrote: post #230

throwaway81523 wrote: post #217

Earlier quoted context omitted.

I've generally been able to deal with embarassing parallelism by just chopping up the input and running multiple processes with GNU Parallel. I haven't needed the multiprocessing module or free threading so far. I believe CPython still relies on various bytecodes to run atomically, which you get automatically with the GIL present. So I wonder if hard-to-reproduce concurrency bugs will keep surfacing in the free-threa…

Async seems fine? What's wrong with it?

Watch this video and maybe you'll understand ;). Warning, NSFW (lots of swearing), use headphones.

https://www.youtube.com/watch?v=bzkRVzciAZg

This is also good:

https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

web search on "colored functions" finds lots of commentary on that article.

Re: Free-threaded CPython is ready to experiment with

#235

Post by dudus » Sat, Jul 13, 2024, 7:03 AM UTC

d0mine wrote: post #196

dagenix wrote: post #131

Earlier quoted context omitted.

Are you writing an extension or Python code? If you are writing Python code, the GIL can already be dropped at pretty much any point and there isn't much way of controlling when. Iirc, this includes in the middle of things like +=. There are some operations that Python defines as atomic, but, as I recall, there aren't all that many. In what way is the GIL preventing races for your use case?

It is not about your code, it is about C extensions you are relying on. Without GIL, you can't even be sure that refcounting works reliably. Bugs in C extensions are always possible. No GIL makes them more likely. Even if you are not the author of C extension, you have to debug the consequences.

Does that mean rewriting all the extensions to Rust? Or maybe CPython itself?

Would that be enough to make Python no gill viable?

Re: Free-threaded CPython is ready to experiment with

#236

Post by vegabook » Sat, Jul 13, 2024, 7:27 AM UTC

Clearly the Python 2 to 3 war was so traumatising (and so badly handled) that the core Python team is too scared to do the obvious thing, and call this Python 4.

This is a big fundamental and (in many cases breaking) change, even if it's "optional".

Re: Free-threaded CPython is ready to experiment with

#237

Post by skissane » Sat, Jul 13, 2024, 7:30 AM UTC

pizza234 wrote: post #231

eigenvalue wrote: post #12

Really excited for this. Once some more time goes by and the most important python libraries update to support no GIL, there is just a tremendous amount of performance that can be automatically unlocked with almost no incremental effort for so many organizations and projects. It's also a good opportunity for new and more actively maintained projects to take market share from older and more established libraries if th…

> using simple threads instead of dealing with the massive overhead and complexity and bugs of using something like multiprocessing. Depending on the domain, the reality can be the reverse. Multiprocessing in the web serving domain, as in "spawning separate processes", is actually simpler and less bug-prone, because there is considerably less resource sharing. The considerably higher difficulty of writing, testing an…

Just the other day I was trying to do two things in parallel in Python using threads - and then I switched to multiprocessing - why? I wanted to immediately terminate one thing whenever the other failed. That’s straightforwardly supported with multiprocessing. With threads, it gets a lot more complicated and can involve things with dubious supportability

Re: Free-threaded CPython is ready to experiment with

#238

Post by wokwokwok » Sat, Jul 13, 2024, 7:33 AM UTC

eigenvalue wrote: post #12

Really excited for this. Once some more time goes by and the most important python libraries update to support no GIL, there is just a tremendous amount of performance that can be automatically unlocked with almost no incremental effort for so many organizations and projects. It's also a good opportunity for new and more actively maintained projects to take market share from older and more established libraries if th…

> there is just a tremendous amount of performance that can be automatically unlocked with almost no incremental effort for so many organizations and projects

This just isn’t true.

This does not improve single threaded performance (it’s worse) and concurrent programming is already available.

This will make it less annoying to do concurrent processing.

It also makes everything slower (arguable where that ends up, currently significantly slower) overall.

This way over hyped.

At the end of the day this will be a change that (most likely) makes the existing workloads for everyone slightly slower and makes the lives of a few people a bit easier when they implement natively parallel processing like ML easier and better.

It’s an incremental win for the ML community, and a meaningless/slight loss for everyone else.

At the cost of a great. Deal. Of. Effort.

If you’re excited about it because of the hype and don’t really understand it, probably calm down.

Mostly likely, at the end of the day, it s a change that is totally meaningless to you, won’t really affect you other than making some libraries you use a bit faster, and others a bit slower.

Overall, your standard web application will run a bit slower as a result of it. You probably won’t notice.

Your data stack will run a bit faster. That’s nice.

That’s it.

Over hyped. 100%.

Re: Free-threaded CPython is ready to experiment with

#239

Post by arp242 » Sat, Jul 13, 2024, 7:35 AM UTC

simonw wrote: post #126

arp242 wrote: post #80

Earlier quoted context omitted.

Managing a farm of virtualenvs and mucking about with my PATH doesn't address the user-installable problem at all. And it seems there's a new tool to try every few months that really will fix all problems this time. And maybe if you're a Python developer working on the code every day that's all brilliant. But most people aren't Python developers, and I just want to try that "Show HN" project or whatnot. Give me a sin…

"Give me a single command I can run. Always. For any project. And that always works." pipx install X

Right so; I'll try that next time. Thanks. I just go by the very prominent "pip install X" on every pypi page (as well as "pip install .." in many READMEs).

Re: Free-threaded CPython is ready to experiment with

#240

Post by wesselbindt » Sat, Jul 13, 2024, 7:37 AM UTC

throwaway81523 wrote: post #234

robertlagrant wrote: post #230

Earlier quoted context omitted.

Async seems fine? What's wrong with it?

Watch this video and maybe you'll understand ;). Warning, NSFW (lots of swearing), use headphones. https://www.youtube.com/watch?v=bzkRVzciAZg This is also good: https://journal.stuffwithstuff.com/2015/02/01/what-color-is-... web search on "colored functions" finds lots of commentary on that article.

I've always found the criticism leveled by the colored functions blog post a bit contrived. Yes, when you replace the words async/await with meaningless concepts I do not care about, it's very annoying to have to arbitrarily mark a function as blue or red. But when you replace the word "aync" with something like "expensive", or "does network calls", it becomes clear that "async/await" makes intrinsic properties about your code (e.g., is it a bad idea to put this call in a loop from a performance perspective) explicit rather than implicit.

In short, "await" gives me an extra piece of data about the function, without having to read the body of the function (and the ones it calls, and the ones they call, etc). That's a good thing.

There are serious drawbacks to async/await, and the red/blue blog post manages to list none of them.

EDIT: all of the above is predicated on the idea that reading code is harder than writing it. If you believe the opposite, then blue/red has a point.