Live data from Hacker News

500 Python Interpreters

izzys.casa

11–20 of 35 posts

Re: 500 Python Interpreters

#11

When they say per interpreter GIL in PEP 684, is that per thread? I wasn't aware of multiple interpreters in one process.

Afaik, you could have multiple CPython interpreters per process before, but the GIL was global to the address space, not a per interpreter instance. I believe the GIL work also put interpreter state behind a single struct instead of a handful of global variables.

This could be totes wrong, all from memory, too tired to fact check.

Re: 500 Python Interpreters

#12

When they say per interpreter GIL in PEP 684, is that per thread? I wasn't aware of multiple interpreters in one process.

No not per thread. Each interpreter is capable of spawning threads, all of which share that interpreter's gil

Re: 500 Python Interpreters

#13
post #11

When they say per interpreter GIL in PEP 684, is that per thread? I wasn't aware of multiple interpreters in one process.

Afaik, you could have multiple CPython interpreters per process before, but the GIL was global to the address space, not a per interpreter instance. I believe the GIL work also put interpreter state behind a single struct instead of a handful of global variables. This could be totes wrong, all from memory, too tired to fact check.

These were two separate and mostly unrelated PEPs but otherwise that's correct.

Re: 500 Python Interpreters

#14
post #11

Earlier quoted context omitted.

Afaik, you could have multiple CPython interpreters per process before, but the GIL was global to the address space, not a per interpreter instance. I believe the GIL work also put interpreter state behind a single struct instead of a handful of global variables. This could be totes wrong, all from memory, too tired to fact check.

These were two separate and mostly unrelated PEPs but otherwise that's correct.

Thanks for corroborating, I'll have to trust you.

Putting interpreter state behind a struct allows for multiple CPython instances per address space and each can no run on their own thread independently, they all get their own GI(L) lock. These interpreter instances would still have their own Global to This interpreter Lock, but not a Global lock for all CPython interpreters running in a process (like it was before with interpreter state in globals).

Unrelated but complementary. We could remove the GIL, making Python multithreaded, but if interpreter state was still in a handful globals, you could have only one interpreter per process.

Again, could be totes wrong.

Re: 500 Python Interpreters

#17

> // 500 interpreter states > static constexpr auto MAXIMUM_STATES = 463; There is a joke here that I’m missing. Does anyone understand what it is?

In the TV show "The Orville" a character has their matter replicator create "500 cigarettes", but it only created 463 after someone counted them all. This is a meme that's been making the rounds for a few weeks in some circles now.

Re: 500 Python Interpreters

#18
So this means modules like concurrent.futures can create a thread pool object (as opposed to processpool) and run in parallel?

Or is the functionality as of 3.13 still limited to low level python embedding applications?

Re: 500 Python Interpreters

#19

> // 500 interpreter states > static constexpr auto MAXIMUM_STATES = 463; There is a joke here that I’m missing. Does anyone understand what it is?

In the TV show "The Orville" a character has their matter replicator create "500 cigarettes", but it only created 463 after someone counted them all. This is a meme that's been making the rounds for a few weeks in some circles now.

Thanks!
Post reply on HN