Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Free-threaded CPython is ready to experiment with

labs.quansight.org

Post reply on HN
Print view View original

311–320 of 398 posts

Re: Free-threaded CPython is ready to experiment with

#311

Post by hot_gril »

j1elo wrote: post #108
BugsJustFindMe wrote: post #45

Earlier quoted context omitted.

Working with asyncio sucks when all you want is to be able to do some things in the background, possibly concurrently. You have to rewrite the worker code using those stupid async await keywords. It's an obnoxious constraint that completely breaks down when you want to use unaware libraries. The thread model is just a million times easier to use because you don't have to change the code.

So, in Rust they had threading since forever and they are now hyped with this new toy called async/await (and all the new problems it brings), while in Python they've had async/await and are now excited to see the possibilities of this new toy called threads (and all its problems). That's funny!

Python is more so in the same boat as Rust. Python asyncio was relatively recent.

Re: Free-threaded CPython is ready to experiment with

#312

Post by [deleted] »

nvy wrote: post #160
j1elo wrote: post #108

Earlier quoted context omitted.

So, in Rust they had threading since forever and they are now hyped with this new toy called async/await (and all the new problems it brings), while in Python they've had async/await and are now excited to see the possibilities of this new toy called threads (and all its problems). That's funny!

Being hyped for is totally on-brand for the Rust community.

[deleted]

Re: Free-threaded CPython is ready to experiment with

#313

Post by hot_gril »

nvy wrote: post #160
j1elo wrote: post #108

Earlier quoted context omitted.

So, in Rust they had threading since forever and they are now hyped with this new toy called async/await (and all the new problems it brings), while in Python they've had async/await and are now excited to see the possibilities of this new toy called threads (and all its problems). That's funny!

Being hyped for is totally on-brand for the Rust community.

That sounds more like Golang (generics)

Re: Free-threaded CPython is ready to experiment with

#314

Post by pdonis »

dagenix wrote: post #200
pdonis wrote: post #188

Earlier quoted context omitted.

> This code could already be run in multiple threads today, with a GIL. Yes. > And it would be broken - in all the same ways it would be broken without a GIL, correct? Yes, but the absence of the GIL would make race conditions more likely to happen. > is your point that removing the GIL will cause people to take non-multithread code and run it in multiple threads without realizing that it is broken in that context? Y…

> Yes, but the absence of the GIL would make race conditions more likely to happen. Does it though? I'm not saying it doesn't, I'm quite curious. Switching between threads with the GIL is already fairly unpredictable from the perspective of pure-Python code. Does it get significantly more troublesome without the GIL? > Yes. They could run it in multiple threads with the GIL today, but as above, race conditions might…

> Switching between threads with the GIL is already fairly unpredictable from the perspective of pure-Python code.

But it still prevents multiple threads from running Python bytecode at the same time: in other words, at any given time, only one Python bytecode can be executing in the entire interpreter.

Without the GIL that is no longer true; an arbitrary number of threads can all be executing a Python bytecode at the same time. So even Python-level operations that only take a single bytecode now must be protected to be thread-safe--where under the GIL, they didn't have to be. That is a significant increase in the "attack surface", so to speak, for race conditions in the absence of thread safety protections.

(Note that this does mean that even multi-threaded code that was race-free with the GIL due to using explicit locks, mutexes, semaphores, etc., might not be without the GIL if those protections were only used for multi-bytecode operations. In practice, whether or not a particular Python operation takes a single bytecode or multiple bytecodes is not something you can just read off from the Python code--you have to either have intimate knowledge of the interpreter's internals or you have to explicitly disassemble each piece of code and look at the bytecode that is generated. Of course the vast majority of programmers don't do that, they just use thread safety protections for every data mutation, which will work without the GIL as well as with it.)

Re: Free-threaded CPython is ready to experiment with

#315

Post by jacob019 »

fuzztester wrote: post #211
jacob019 wrote: post #172

Earlier quoted context omitted.

Bashing Python in Python development thread is not tactful.

it may not be tactful but it sure is factful (even if that last word is ungrammatical, at least the sentence is poetical). he he he. jfc. guido knows ;), this thread is getting weirder and weirder, creepier and creepier. are you literally implying that I should lie about known facts about python slowness? "tactful" my foot. then I guess the creators of PyPy and Unladen Swallow (the latter project was by Google) were/…

tactful - showing skill and sensitivity in dealing with people. Right or wrong, your comments do not contribute to the discussion. Open source development is about collaboration. Your irrelevant comments are disrespectful to all the hard working developers who have been working together to push the limits of what is possible with what may be the most widely used programming language of our time. I use Python for it's ergonomics and community, not for speed. Removing the GIL has been something we've been yearning for since the Python 2 days. And the speed has improved drastically in recent years with each release. You don't win a prize for being right.

Re: Free-threaded CPython is ready to experiment with

#316

Post by imtringued »

simonw wrote: post #67
arp242 wrote: post #56

Earlier quoted context omitted.

I guess that depends from your perspective. I'm not a Python developer, but like many people I do want to run Python programs from time to time. I don't really know Rust, or Cargo, but I never have trouble building any Rust program: "cargo build [--release]" is all I need to know. Easy. Even many C programs are actually quite easy: "./configure", "make", and optionally "make install". "./configure" has a nice "--help…

"I don't want a bunch of venvs" That's your problem right there. Virtual environments are the Python ecosystem's solution to the problem of wanting to install different things on the same machine that have different conflicting requirements. If you refuse to use virtual environments and you install more than one separate Python project you're going to run into conflicting requirements and it's going to suck. Have you…

Pythons venvs are a problem to the solution of solving the dependency problem. Consider the following: it is not possible to relocate venvs. In what universe does this make sense? Consider a C++ or Rust binary that would only run when it is placed in /home/simonw/.

Re: Free-threaded CPython is ready to experiment with

#317

Post by galdosdi »

jolux wrote: post #9
janice1999 wrote: post #7

Earlier quoted context omitted.

Not sure what this list means, there are successful languages without these feature. Also Python 3.13 [1] has an optional JIT [2], disabled by default. [1] https://docs.python.org/3.13/whatsnew/3.13.html [2] https://peps.python.org/pep-0744/

The successful languages without efficient dependency management are painful to manage dependencies in, though. I think Python should be shooting for a better package management user experience than C++.

The shitshow that is python tooling is one of the reasons I prefer java jobs to python jobs when I can help it. Java got this pretty right years and years and years earlier. Why are python and javascript continuing to horse around playing games?

Re: Free-threaded CPython is ready to experiment with

#318

Post by imtringued »

Galanwe wrote: post #213
arp242 wrote: post #56

Earlier quoted context omitted.

I guess that depends from your perspective. I'm not a Python developer, but like many people I do want to run Python programs from time to time. I don't really know Rust, or Cargo, but I never have trouble building any Rust program: "cargo build [--release]" is all I need to know. Easy. Even many C programs are actually quite easy: "./configure", "make", and optionally "make install". "./configure" has a nice "--help…

Maybe I am biased, because I learned these things so long ago and I don't realize that it's a pain to learn. But what exactly is so confusing about virtualenvs ? They really not that different from any other packaging system like JS or Rust. The only difference is instead of relying on your current directory to find the the libraries / binaries (and thus requiring you to wrap binaries call with some wrapper to search…

Only python demands you to source an activation script before doing anything.

Re: Free-threaded CPython is ready to experiment with

#319

Post by imtringued »

ngoldbaum wrote: post #24
mihaic wrote: post #19

Does anyone know if there is more serious single threaded performance degradation (more than a few percent for instance)? I couldn't find any benchmarks, just some generic reassurance that everything is fine.

Right now there is a significant single-threaded performance cost. Somewhere from 30-50%. Part of what my colleague Ken Jin and others are working on is getting back some of that lost performance by applying some optimizations. Expect single-threaded performance to improve for Python 3.14 next year.

That kind of negates the whole purpose of multi threading. An application running on two cores might end up slower, not faster. We know that the python developers are kind of incompetent when it comes to performance, but the numbers you are quoting are so bad they probably aren't correct in the first place.

Re: Free-threaded CPython is ready to experiment with

#320

Post by thomasjudge »

ignoramous wrote: post #302
bausgwi678 wrote: post #249

Earlier quoted context omitted.

Using multiple processes is simpler in terms of locks etc, but python libraries like multiprocessing or even subprocess.popen[1] which make using multiple processes seem easy are full of footguns which cause deadlocks due to fork-safe code not being well understood. I’ve seen this lead to code ‘working’ and being merged but then triggering sporadic deadlocks in production after a few weeks. The default for multiproce…

> The default for multiprocessing is still to fork (fortunately changing in 3.14) If I may: Changing from fork to what?

"In Python 3.14, the default will be changed to either “spawn” or “forkserver” (a mostly safer alternative to “fork”)."

- https://pythonspeed.com/articles/python-multiprocessing/

Post reply on HN