Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A viable solution for Python concurrency

lwn.net

Post reply on HN
Print view View original

21–30 of 366 posts

Re: A viable solution for Python concurrency

#21

Post by calpaterson »

nerdponx wrote: post #8
Animats wrote: post #4

Or you could just use PyPy, which uses a garbage collector, does more compile-time analysis, and runs much faster. CPython is a naive interpreter, like original JavaScript. There's been progress since then.

PyPy being stuck on 3.7 hurts. If 3.8 support comes out soon, I'll be happy to switch for general-purpose work. 3.9 would be even nicer, to support the type annotation improvements. I donate every month, but I'm just an individual donating pocket change; it'd be great to see some corporate support for PyPy.

There are very few new features in 3.8.

It is a much less important release (for features) than 3.7, which for example added dataclasses and lots of typing and asyncio stuff.

The most significant change in 3.8 is a notoriously controversial new infix operator. Even it's supporters would say that it's a niche usecase.

Re: A viable solution for Python concurrency

#24

Post by calpaterson »

tbabb wrote: post #19
ferdowsi wrote: post #11

If this effort succeeds (and I hope it does) now Python developers will need to contend with the event-loop albatross of asyncio and all of its weird complexity. In an alternate Python timeline, asyncio was not introduced into the Python standard library, and instead we got a natively supported, robust, easy-to-use concurrency paradigm built around green/virtual threading that accommodates both IO and CPU bound work.

What specifically is the problem with asyncio? I quite like using it, so I'm curious if there's some aspect that makes it unsustainable?

The key disadvantage is largely that it bifurcates the library base. Async libraries and sync libraries co-exist uneasily in the same program.

For nearly every popular library there is now a (usually inferior, less robust) async one. The benefits of Linus' Law are reduced.

Re: A viable solution for Python concurrency

#25

Post by phkahler »

>> If that bit is set, the interpreter doesn't bother tracking references for the relevant object at all. That avoids contention (and cache-line bouncing) for the reference counts in these heavily-used objects. This "optimization" actually slows single-threaded accesses down slightly, according to the design document, but that penalty becomes worthwhile once multi-threaded execution becomes possible.

Was going to say do the opposite. Set the bit if you want counting and then modify the increment and decrement to add or subtract the bit, thereby eliminating condition checking and branching. But it sounds like the concern is cache behavior when the count is written. Checking the bit can avoid any modification at all.

Re: A viable solution for Python concurrency

#26

Post by BiteCode_dev »

ferdowsi wrote: post #11

If this effort succeeds (and I hope it does) now Python developers will need to contend with the event-loop albatross of asyncio and all of its weird complexity. In an alternate Python timeline, asyncio was not introduced into the Python standard library, and instead we got a natively supported, robust, easy-to-use concurrency paradigm built around green/virtual threading that accommodates both IO and CPU bound work.

asyncio is not a competition to threads, it's complementary.

In fact, it's a perfectly viable strat in python to have several processes, each having several threads, each having an event loop.

And it will still be so, once this comes out. You will certainly use threads more, and processes less, but replacing 1000000 coroutines by 1000000 system threads is not necessarily the right strategy for your task. See nginx vs apache.

Re: A viable solution for Python concurrency

#27

Post by a1369209993 »

misnome wrote: post #2

> This "optimization" actually slows single-threaded accesses down slightly, according to the design document, but that penalty becomes worthwhile once multi-threaded execution becomes possible. My understanding was that CPython viewed any single-threaded performance regression as a blocker to GIL-removal attempts, regardless of if other work by the developer has sped up the interpreter? This article seems to somewha…

> and one day everyone will realise it

No? Why would we think that? There are people who willingly use java; compared to that the problems with python 3 are downright non-obvious as long as you never need to work with things like non-Unicode text.

Re: A viable solution for Python concurrency

#28

Post by fullstop »

tbabb wrote: post #19
ferdowsi wrote: post #11

If this effort succeeds (and I hope it does) now Python developers will need to contend with the event-loop albatross of asyncio and all of its weird complexity. In an alternate Python timeline, asyncio was not introduced into the Python standard library, and instead we got a natively supported, robust, easy-to-use concurrency paradigm built around green/virtual threading that accommodates both IO and CPU bound work.

What specifically is the problem with asyncio? I quite like using it, so I'm curious if there's some aspect that makes it unsustainable?

I like using it as well, but I've been bit several times by having runtime exceptions completely swallowed.

Re: A viable solution for Python concurrency

#30

Post by masklinn »

misnome wrote: post #2

> This "optimization" actually slows single-threaded accesses down slightly, according to the design document, but that penalty becomes worthwhile once multi-threaded execution becomes possible. My understanding was that CPython viewed any single-threaded performance regression as a blocker to GIL-removal attempts, regardless of if other work by the developer has sped up the interpreter? This article seems to somewha…

> My understanding was that CPython viewed any single-threaded performance regression as a blocker to GIL-removal attempts, regardless of if other work by the developer has sped up the interpreter?

Previous GILectomy attempts incurred significant single-threaded performance penalties, on on the order of 50% or above. If Gross's work yields low single-digit performance penalty it's pretty likely to be accepted as this is the sort of impacts which can happen semi-routinely as part of interpreter updates.

The complete breakage of C extensions would be a much bigger issue.

Post reply on HN