Live data from Hacker News

What's up, Python? The GIL removed, a new compiler, optparse deprecated

bitecode.dev

301–302 of 302 posts

Re: What's up, Python? The GIL removed, a new compiler, optparse deprecated

#301

Earlier quoted context omitted.

> So, all synchronization requires dealing with locks, mutexes, condition variables... As always, by far the best way to interact between threads is to use thread-safe queues (AKA message passing). Luckily, Python has one of those [1]. No complicated synchronisation needed. [1] https://docs.python.org/3/library/queue.html

That's just completely missing the point of threads... but that wouldn't be the first nor the hundreds stupid thing found in Python's documentation. The reason to want threads is to be able to share memory. That's literally why they were created. If you are sending messages instead of sharing memory, you don't need threads. You need something like Erlang processes. The problem is that people who wrote Python never ha…

The object going onto the queue is the shared memory. The queue itself is essentially a fancy type of lock.

Yes you could use multiple processes, but you have the extra expense of serialisation, or you could use shared memory but you'd have to administer that and you'd still have the expense of context switches. And inevitably, yes, there is some actual shared state like a logger and modules that you've loaded, which again would be a pain in multiple processes.

You can call a Python thread plus a queue an Erlang process if you like, or say that I should use Erlang processes instead. But the fact is, the Python version works perfectly well for many problems. It does all the things that you typically need if threads: shares state (via the queues), let's you concurrently use the CPU (via C libraries that release the GIL – but no GIL would be even better), and writing blocking IO if you wish. Not missing the point at all.

The developers of Python didn't "suck as programmers" and it doesn't help your point to claim they do. Guido choose to use the GIL because he was OK with multiple threads but not at the expense of single thread performance, and no one showed any solution to that that beats the GIL – until now. (Personally I think the trade off was wrong, and a small hit to single threaded performance would have been with it. But that's different from being ignorant to the fact there was an actual reason.)

Re: What's up, Python? The GIL removed, a new compiler, optparse deprecated

#302

Earlier quoted context omitted.

Multithreading is hard but once you have been doing it a while, it becomes easy and most importantly, it’s stable. When you have to deal with processes, there’s a lot of external factors out of your control because processes are much more visible and carry a lot of extra baggage. Hard multithreading problems are fun. Hard multi-process problems are just tedious.

As I understand it on Linux processes and threads are implemented in almost the same way, just that threads share memory. I've heard it said several times that the idea that processes are "heavier" is a bit of a myth. I guess they need to allocate heap space and threads don't. I'm not an expert, just mentioning because it sounded like you might be believing something which is at odds with what people say about proces…

I'm not a Linux kernel dev but I think this is true! Not sure what's up with the downvotes.

You can create a process/thread chimera with certain system calls, and get something that is in-between a thread and process if you want, which is neat but maybe not that useful.

Creating processes on Linux is actually much faster than people seem to realize. I can spawn at least a few thousand a second from a quick test of spawning bash instances.

Post reply on HN