Live data from Hacker News

A viable solution for Python concurrency

lwn.net

11–20 of 366 posts

Re: A viable solution for Python concurrency

#11
If this effort succeeds (and I hope it does) now Python developers will need to contend with the event-loop albatross of asyncio and all of its weird complexity.

In an alternate Python timeline, asyncio was not introduced into the Python standard library, and instead we got a natively supported, robust, easy-to-use concurrency paradigm built around green/virtual threading that accommodates both IO and CPU bound work.

Re: A viable solution for Python concurrency

#12
post #4

Or you could just use PyPy, which uses a garbage collector, does more compile-time analysis, and runs much faster. CPython is a naive interpreter, like original JavaScript. There's been progress since then.

lots of people need C extensions, which you can't* have on pypy.

*: mostly true

Re: A viable solution for Python concurrency

#14
post #2

> This "optimization" actually slows single-threaded accesses down slightly, according to the design document, but that penalty becomes worthwhile once multi-threaded execution becomes possible. My understanding was that CPython viewed any single-threaded performance regression as a blocker to GIL-removal attempts, regardless of if other work by the developer has sped up the interpreter? This article seems to somewha…

From the article:

> Gross has also put some significant work into improving the performance of the CPython interpreter in general. This was done to address the concern that has blocked GIL-removal work in the past: the performance impact on single-threaded code. The end result is that the new interpreter is 10% faster than CPython 3.9 for single-threaded programs.

Re: A viable solution for Python concurrency

#15
post #6

Why not using something trio or curio? They are quite easy to learn, very powerful and have an approach similar to channel in golang.

Those are not multithreading, they are asynchronous io which is different. With asynchronous io in Python the only concurrency/parallelism you can do is for IO.

Multithreading in Python currently has the same limitation though but it needn't.

Re: A viable solution for Python concurrency

#16
post #4

Or you could just use PyPy, which uses a garbage collector, does more compile-time analysis, and runs much faster. CPython is a naive interpreter, like original JavaScript. There's been progress since then.

Pypy is still single threaded. https://doc.pypy.org/en/latest/faq.html#does-pypy-have-a-gil...

This work is super exciting! Can pypy use the same recipe to offer true parallelism plus the jit??

Will be really interesting to see what pypy devs think of this work and how they might also lever it!

Re: A viable solution for Python concurrency

#17
post #9
post #2

> This "optimization" actually slows single-threaded accesses down slightly, according to the design document, but that penalty becomes worthwhile once multi-threaded execution becomes possible. My understanding was that CPython viewed any single-threaded performance regression as a blocker to GIL-removal attempts, regardless of if other work by the developer has sped up the interpreter? This article seems to somewha…

It was Guido's requirement that GIL removal not degrade single threaded performance at all, but in the talk I attend at PyCon 2019, the speaker mentioned nothing about qualifications on that. Guido's restriction was presented, quite reasonably, as "no one should have to suffer because of removing the GIL". So a net break-even or performance improvement is fine. And on top of that, Guido has retired now, and the steer…

Guido has replied to Gross's announcement to observe that his performance improvements are not tied to removing the GIL and could be accepted separately. But he doesn't reject Gross's work outright, and if the same release that includes the GIL removal also delivers a concrete performance upgrade, I suspect that Guido would be fine with it. His concern is, after all, practical, to do with the actual use of python and not some architecture principle.

Re: A viable solution for Python concurrency

#18
post #2

> This "optimization" actually slows single-threaded accesses down slightly, according to the design document, but that penalty becomes worthwhile once multi-threaded execution becomes possible. My understanding was that CPython viewed any single-threaded performance regression as a blocker to GIL-removal attempts, regardless of if other work by the developer has sped up the interpreter? This article seems to somewha…

From the article: > Gross has also put some significant work into improving the performance of the CPython interpreter in general. This was done to address the concern that has blocked GIL-removal work in the past: the performance impact on single-threaded code. The end result is that the new interpreter is 10% faster than CPython 3.9 for single-threaded programs.

Sorry, to be clear, I missed your point "regardless of if other work by the developer has sped up the interpreter". That's fair, though my personal opinion is that that seems like an incredibly high bar for any language.

Re: A viable solution for Python concurrency

#19

If this effort succeeds (and I hope it does) now Python developers will need to contend with the event-loop albatross of asyncio and all of its weird complexity. In an alternate Python timeline, asyncio was not introduced into the Python standard library, and instead we got a natively supported, robust, easy-to-use concurrency paradigm built around green/virtual threading that accommodates both IO and CPU bound work.

What specifically is the problem with asyncio? I quite like using it, so I'm curious if there's some aspect that makes it unsustainable?

Re: A viable solution for Python concurrency

#20
I'm going to assume that there is a reason that this isn't a switch control, so that the default is a single-threaded program and the programmer needs to state explicitly that this one will be multi-threaded, upon which the interpreter changes into the atomic mode for the rest of execution?
Post reply on HN