Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A viable solution for Python concurrency

lwn.net

Post reply on HN
Print view View original

261–270 of 366 posts

Re: A viable solution for Python concurrency

#261

Post by otterley »

Mehdi2277 wrote: post #220
otterley wrote: post #127

Earlier quoted context omitted.

Don't get me wrong; I'm not suggesting that anyone dump Python altogether to switch to a different language for any arbitrary project or purpose. Many businesses I work with use different languages for different components or applications, using the network or storage (or even shared memory) to intercommunicate when necessary. The right tool for the job, as it were.

I use python mostly for numpy/tensorflow/etc. The machine learning ecosystem. That ecosystem cares a lot about multithreaded performance. So historically answer has been write c/c++ and then bind to python. This work is mainly motivated by libraries like that wanting to write less c extensions and be able to just write python and still have proper thread performance.

As I mention elsewhere, I think what they care about is parallelism, not concurrency, and you don’t need threads to attain that efficiently.

Re: A viable solution for Python concurrency

#262

Post by modeless »

Threads are certainly important, but I have to say that I found the multiprocessing package to work very well. I think a lot of the things people think they need threads for would actually be better with multiprocessing instead. Memory protection is good! Shared memory is still available and explicit sharing of just what you need is better in a lot of ways than implicit sharing of everything.

I will be glad if the GIL is fixed but I think people reach for threads too quickly and too often.

Re: A viable solution for Python concurrency

#263

Post by sandGorgon »

btown wrote: post #252
js2 wrote: post #212

Earlier quoted context omitted.

This is why you run one process per core, and you'll typically have something like nginx+uWSGI distribute requests across them. I use this combination with https://falconframework.org/ and boto3 to spool HTTP POST requests to S3 and SQS and am pretty happy with it. uWSGI supports gevent https://uwsgi-docs.readthedocs.io/en/latest/Gevent.html Falcon also benefits from Cython acceleration. It's been a while but at the…

Yep, we use Gunicorn in full gevent mode, tuned to spawn and route to a gevent-patched Django process per core, each of which can handle as many concurrent requests as will fit in (its slice of) RAM. A far cry from the one-request-per-thread days of yore!

you use gunicorn as a reverse proxy - it seems that you are not running Gunicorn to spawn django processes is it ?

I'm wondering how would this work in a kubernetes/docker world.

Re: A viable solution for Python concurrency

#264

Post by rtpg »

garmaine wrote: post #249
rtpg wrote: post #243

Earlier quoted context omitted.

`.decode` doesn't work on strings because you decode bytes! PYthon has UTF strings! I agree about the annoyance of the codec modules, and stuff like the urllib reorg happening on 3.0 instead of shifting it to later releases (and self-owns like the u prefix). But if you're calling `decode` on a string you have a bug in hiding, from the moment that your "code that handles encodings transparently" blows up when someone…

So? `.decode(“hex”)` still makes sense on a string, and was >95% of the encode/decode calls I had.

Hex-encoding works off of bytes… like if a character has a different byte representation between two encodings you need to specify in both directions

Even the Python 2 strategy for hex might lead to your text getting garbled if you’re using anything not représentable in ASCII (roughly).

Re: A viable solution for Python concurrency

#265

Post by Mehdi2277 »

otterley wrote: post #260
Mehdi2277 wrote: post #221

Earlier quoted context omitted.

Yes they do. I’ve written cython/numba as work arounds before. A lot of times if you need a small operation done many times the multiprocessor overhead is bad, but writing a pure python for loop over numpy/other tensors is awful for performance. The answer historically has been c/c++ and bind to python. This work is mainly motivated by one of those libraries wanting to write less c++ bindings and be able to do operat…

This doesn’t sound right. I think you’re mixing up parallelism/multi-core/linear resource scalability with concurrency. Usually you want high concurrency to handle multiplexing events where a thread or process per client would be waiting idle the majority of the time. ML is compute bound so by definition there wouldn’t be idle threads. And you can already get multicore work done in Python by simply using its venerabl…

I'm familiar with both. The person who's leading the GIL removal is one of the main authors of one of the two leading ML libraries in python (pytorch) and his motivation is for primarily performance for ML. He explicitly talks about multiprocessing and while it does often work well, there are several situations where it causes issues. I think best thing is to look at his arguments against it, https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsD...

A couple issues with multiprocessing is fork is often likely to dead lock especially with cuda and tensorflow. tensorflow sessions aren't even fork safe and while you can fork before making tensorflow it is a restriction to be careful of. It's also comes with heavier cost for smaller tasks that you want to be short and interactive but are still very compute heavy. Relevant quote for that, "Starting a thread takes ~100 µs, while spawning a sub-process takes ~50 ms (50,000 µs) due to Python re-initialization."

Communication/sharing of memory is also more expensive between processes than with threads. One key quote,

"For example, in PyTorch, which uses multiprocessing to parallelize data loading, copying Tensors to inter-process shared-memory is sometimes the most expensive operation in the data loading pipeline."

My experience has been gpus actually compute fast enough that sometimes memory bandwidth becomes a bottleneck and making memory sharing cheaper becomes very relevant. Data transfer costs are pretty noticeable and something to minimize. I've seen a decent amount of interest in zero deserialization formats from this.

edit: Another example if you examine intel's c++ library for ML operations onednn the implementation is not process heavy. It is based on multithreading. I generally see threading as main primitive for parallelism in C++ libraries even though multiprocessing is certainly an option they can take. You will often find single operations (like one matrix multiplication) have implementations that use multiple threads. For individual operations you aren't going to want to use multiple processes to speed them up. That's why tensorflow has both an inter operation thread count and an intra operation thread count when configured.

Re: A viable solution for Python concurrency

#266

Post by sigotirandolas »

globular-toast wrote: post #181
lucb1e wrote: post #147

Earlier quoted context omitted.

For a minute I thought I finally found someone else who likes the GIL, but then you said content about . Programs that just divide up work across processes are much easier to write without introducing obscure bugs due to the lack of atomicity. I'm definitely excited for a GIL-less python, even if it's a rare scenario where it makes sense to try to do performant code in python in the first place rather than offloading…

> Programs that just divide up work across processes are much easier to write without introducing obscure bugs due to the lack of atomicity. You often don't even need to do this yourself. GNU parallel is the way to go for dividing work up amongst CPU cores. Why reinvent the wheel? I agree with you that threads are talked about way more than they should be. It's like all programmers learn this one simple rule: to be f…

I don't think the parent was proposing reinventing the wheel, Python has straightforward process parallelism support in the 'multiprocessing' library and for Python that's generally a better idea than GNU Parallel, IMO.

Re: A viable solution for Python concurrency

#267

Post by adwn »

throwaway81523 wrote: post #197
dagenix wrote: post #185

Earlier quoted context omitted.

IMO, it's a good thing that decode() doesn't on strings in Python 3.

There is a case to be made either way. The point is that it's yet another quirky difference that pops up sometimes, and using python3 keeps bringing up a stream of such issues. It does seem in the spirit of Python's duck typing to be able to say '616263'.decode('hex') and get 'abc', and that does work fine in py2. Try it in py3 and you get a type error. So ok, convert the string to bytes, e.g. by saying b'616263' ins…

So, like this?

    codecs.decode(b'616263', 'hex')
If that's correct, what's so hard/bad about this?

Re: A viable solution for Python concurrency

#268

Post by adwn »

globular-toast wrote: post #181
lucb1e wrote: post #147

Earlier quoted context omitted.

For a minute I thought I finally found someone else who likes the GIL, but then you said content about . Programs that just divide up work across processes are much easier to write without introducing obscure bugs due to the lack of atomicity. I'm definitely excited for a GIL-less python, even if it's a rare scenario where it makes sense to try to do performant code in python in the first place rather than offloading…

> Programs that just divide up work across processes are much easier to write without introducing obscure bugs due to the lack of atomicity. You often don't even need to do this yourself. GNU parallel is the way to go for dividing work up amongst CPU cores. Why reinvent the wheel? I agree with you that threads are talked about way more than they should be. It's like all programmers learn this one simple rule: to be f…

> GNU parallel is the way to go for dividing work up amongst CPU cores. Why reinvent the wheel?

Because most problems are not the embarrassingly parallel kind suitable for use with GNU parallel. For example, any problems that require some communication between the individual tasks.

Re: A viable solution for Python concurrency

#269

Post by throwaway81523 »

slaymaker1907 wrote: post #250
throwaway81523 wrote: post #59

Earlier quoted context omitted.

Yes, I have a big sense of tragedy about Python 3. Python should run on something like (or maybe the actual) Erlang BEAM with lightweight isolated processes. All my threaded Python code is written using that style anyway (threads communicating through synchronized queues) and I've almost never needed traditional shared mutable objects. Maybe completely never, but I'm not sure about a certain program any more. Added:…

Yeah, I'd agree it's kind of stupid to use OS threads if you are going to have a GIL. It does make the implementation simpler, but it comes at tremendous cost to IO bound programs. If you are actually trying to do computationally intensive work, you should really be using multiple processes instead of threads in a language with GC. When writing a UI, moving work to a separate thread can still lag because GC will also…

The GIL is mostly a problem for cpu bound programs. GC pauses in Python really aren't much of an issue, partly because Python uses refcounting (ironically the cause of the GIL) so it usually frees stuff as it goes along, keeping pauses pretty short. Even with a real GC though, I think it's usually not much of a problem. I've written Python gui's in multi-threaded (but not too compute intensive) apps on quite slow processors by today's standards, and there wasn't much of a problem in UI responsiveness.

I think it's important to remember the difference between parallelism (trying to compute faster by using multiple cpu's simultaneously) and concurrency (communicating on multiple communication channels that are sending stuff non-deterministically). The GIL gets in the way of parallelism but except in extreme cases, it doesn't interfere with concurrency.

Re: A viable solution for Python concurrency

#270

Post by throwaway81523 »

beebmam wrote: post #213
throwaway81523 wrote: post #59

Earlier quoted context omitted.

Yes, I have a big sense of tragedy about Python 3. Python should run on something like (or maybe the actual) Erlang BEAM with lightweight isolated processes. All my threaded Python code is written using that style anyway (threads communicating through synchronized queues) and I've almost never needed traditional shared mutable objects. Maybe completely never, but I'm not sure about a certain program any more. Added:…

I think some people are upset about the irony/sarcasm here, and many people don't appreciate ironic/sarcastic posts (but I do!)

I was being quite literal, not sarcastic. I really do wish that Python had switched to a BEAM like architecture at the Python 3 transition. There is no way that can possibly happen now.
Post reply on HN