91–100 of 111 posts
Earlier quoted context omitted.
Why is it unsafe?
In general only the thread calling fork() gets forked, so unless you call exec() soon after, there are a lot of complications with signals, shared memory.
One thing I'm curious about here is the operational impact. In production systems we often see Python services scaling horizontally because of the GIL limitations. If true parallelism becomes common, it might actually reduce the number of containers/services needed for some workloads. But that also changes failure patterns — concurrency bugs, race conditions, and deadlocks might become more common in systems that wer…
A lot of that has already been solved for by scaling workers to cores along with techniques like greenlets/eventlets that support concurrency without true multithreading to take better advantage of CPU capacity.
Unless you mean you have multiple worker processes (or GIL-free threads).
Earlier quoted context omitted.
Qt does exist. It's not difficult.
Qt costs serious money if you go commercial. That might not be important for a hobby project, but lowers the enthusiasm for using the stack since the big players won't use it unless other considerations compel them.
Earlier quoted context omitted.
I thought libraries had to explicitly opt in to no GIL via a macro or constant or something in C
GP is a clanker spouting off a lot of random nonsense. Edit: Never mind. If it walks like a duck and talks like a duck...
The stream of posts that resemble copy-paste from gemini is really not improving the site IMO. I can just go query it myself thanks.
Earlier quoted context omitted.
In general only the thread calling fork() gets forked, so unless you call exec() soon after, there are a lot of complications with signals, shared memory.
What are the complications? A single thread with its own process sandbox with everything from the parent is exactly what I'd expect coming from C land. Are the complications you refer to specific to the python VM or more general?
An imperfect solution is to require every mutex created to be accompanied by some pthread_atfork, but libraries don’t do that unless forking is specifically requested. In other words, if you don’t control the library you can’t fork.
Earlier quoted context omitted.
If you have enough discipline to make sure you only create threads after all the forking is done, then sure. But having such discipline is harder than just forbidding fork or forbidding threads in your program. It turns a careful analysis of timing and causality into just banning a few functions.
Can't you check what threads are active at the time you fork?
Earlier quoted context omitted.
In general only the thread calling fork() gets forked, so unless you call exec() soon after, there are a lot of complications with signals, shared memory.
What are the complications? A single thread with its own process sandbox with everything from the parent is exactly what I'd expect coming from C land. Are the complications you refer to specific to the python VM or more general?
Earlier quoted context omitted.
It seems like ai generated stuff to me, the whole history is eerily identical
The llm accusations go out of hand nowadays. Cant see any typical AI slop here.
Pangram is pretty reliable and shows 100% AI.
Earlier quoted context omitted.
This is surely why Facebook was interested in funding this work. It is common to have N workers or containers of Python because you are generally restricted to one CPU core per Python process (you can get a bit higher if you use libs that unlock the GIL for significant work). So the only scaling option is horizontal because vertical scaling is very limited. The main downside of this was memory usage. You would have t…
> The main downside of this was memory usage. You would have to load all of your code and libraries N types and in-process caches would become less effective. You can load modules and then fork child processes. Children will share memory with each other (if they need to modify any shared memory, they get copy-on-write pages allocated by the kernel) and you'll save quite a lot on memory.
Earlier quoted context omitted.
What are the complications? A single thread with its own process sandbox with everything from the parent is exactly what I'd expect coming from C land. Are the complications you refer to specific to the python VM or more general?
Even treating the process as read only after forking is potentially fraught. What if a background thread is mutating some data structure? When it forks the data structure might be internally inconsistent because the work to finish the mutation might not be completed. Imagine there are locks held by various threads when it dies, trying to lock those in the child might deadlock or even worse. There's tons of these type…
Conceptually fork "just" noncooperatively preempts and kills all other threads. Use accordingly. Yes it's a giant footgun but then so is all low level "unmanaged" concurrency.