91–100 of 302 posts
Historically I’ve written several services that load up some big datastructure (10s or 100s of GB), then expose an HTTP API on top of it. Every time I’ve done a quick implementation in Python of a service that then became popular (within a firm, so 100s or 1000s of clients) I’ve often ended up having to rewrite in Java so I can throw more threads at servicing the requests (often CPU heavy). I may have missed somethin…
Earlier quoted context omitted.
I feel like orchestrating thousands of GPUs is such a niche use case that it’s fair to expect the people wanting to do it to learn a more suited language, rather than ruining Python for everyone else.
I notice you used the strong emotional word "ruining" when talking about the effect on Python of this change. Why do you believe an obscure runtime concurrency detail which will make more things possible will "ruin" the language? Now match and :=? Those definitely ruin the language. ;-) But seriously, relax, nothing bad is happening here. It's not just people who have to use the torch launcher who have been bitten by…
It is not obscure. It will make it much more difficult to write native-code extensions which is IMO the whole point of Python.
Historically I’ve written several services that load up some big datastructure (10s or 100s of GB), then expose an HTTP API on top of it. Every time I’ve done a quick implementation in Python of a service that then became popular (within a firm, so 100s or 1000s of clients) I’ve often ended up having to rewrite in Java so I can throw more threads at servicing the requests (often CPU heavy). I may have missed somethin…
> I may have missed something but I couldn’t figure out how to get the multi-threaded performance out of Python Multiprocessing. The answer is to use the python multiprocessing module, or to spin up multiple processes behind wsgi or whatever. > Historically I’ve written several services that load up some big datastructure (10s or 100s of GB), then expose an HTTP API on top of it. Use the python multiprocessing module…
Nowadays numba is usually a better solution for when you want to run some computationally expensive python code that itself calls numpy, etc.
For the parent commenter's use case though that wouldn't be a great solution either. In general, Python does not have an optimal way of operating on a shared data structure across OS threads and certainly not in a way that doesn't require forking the interpreter.
Earlier quoted context omitted.
I am not too deeply experienced with Python so forgive my ignorance. But I am curious to understand why you were not able to utilize the concurrency tools provided in Python. A quick google search gave me these relevant resources 1. An intro to threading in Python ( https://realpython.com/intro-to-python-threading/#conclusion... ) 2. Speed Up Your Python Program With Concurrency ( https://realpython.com/python-concur…
Re (3): asyncio does not give you a boost for CPU bound tasks. It's a single-threaded, cooperative multi-tasking system that can (if you're IO bound) give you a performance boost.
You can absolutely send stuff to a thread pool executor or process pool executor and then never await the returned value/never have it "return until interrupted, but the issues with shared memory (or really, the lack thereof in comparison to ex C) are still present to my understanding.
Then again, I mean you can always spin up a sqllite server or something on the same machine, but that's stupid heavy and more of a workaround than a solution. Super excited for nogil.
https://docs.python.org/3/library/concurrent.futures.html#co...
> tools like pip-run already support running a script for which you have the deps described with such comments > Packages are installed in a temporary virtual env and deleted after the run, like npx used to do for the JS world. Is it efficient? Download packages, install them only to delete several seconds later. Wastes precious SSD cells.
It wasn't long ago that a developer I was working with seemed to have entirely not comprehended the idea when I asked why he was searching for and downloading a dozen-MB PDF just to open (i.e. delete when closed) every time he wanted to look up one thing in it! I accumulate documentation for a project and keep most of it open throughout; I thought that was a usual thing to do, but apparently others will go online to search for that information every single time, then close the browser and reopen it whenver they need to look up something else.
More publicly, it's also not long ago that Docker, and more relevantly, PyPI, have been getting worried about their bandwidth usage: https://news.ycombinator.com/item?id=24262757 https://news.ycombinator.com/item?id=27205586
Historically I’ve written several services that load up some big datastructure (10s or 100s of GB), then expose an HTTP API on top of it. Every time I’ve done a quick implementation in Python of a service that then became popular (within a firm, so 100s or 1000s of clients) I’ve often ended up having to rewrite in Java so I can throw more threads at servicing the requests (often CPU heavy). I may have missed somethin…
My tip for this is Node.js and some stream processing lib like Highland. You can get ridiculous IO parallelism with a very little code and a nice API. Python just scales terribly, no matter if you use multi-process or not. Java can get pretty good perf, but you'll need some libs or quite a bit of code to get nonblocking IO sending working well, or you're going to eat huge amounts of resources for moderate returns. No…
Did I miss something? Does nodes/highland have good shared memory semantics these days?
I've always felt the best analogy to python concurrency was (node)js, but I admittedly haven't kept up all that well.
Historically I’ve written several services that load up some big datastructure (10s or 100s of GB), then expose an HTTP API on top of it. Every time I’ve done a quick implementation in Python of a service that then became popular (within a firm, so 100s or 1000s of clients) I’ve often ended up having to rewrite in Java so I can throw more threads at servicing the requests (often CPU heavy). I may have missed somethin…
> I may have missed something but I couldn’t figure out how to get the multi-threaded performance out of Python Multiprocessing. The answer is to use the python multiprocessing module, or to spin up multiple processes behind wsgi or whatever. > Historically I’ve written several services that load up some big datastructure (10s or 100s of GB), then expose an HTTP API on top of it. Use the python multiprocessing module…
I assume mod_wsgi under apache was not the answer here due to memory constraints. That being said, why not serve from disk and use redis for a cache. This should work well unless the queries had high cardinality.
From reading the thread on HN the other day, it sounds like removing the GIL isn't really of much value. Maybe for somewhat obscure multithreading cases. Is that right?
Correct, it will help with CPU-limited, embarrassingly parallelizable problems... which are much less common than you think.
Earlier quoted context omitted.
The opposite, writing multithreaded code will get harder because you'll likely need to handle concurrency issues yourself that the GIL previously avoided. But, the tradeoff is that multithreaded programs could now actually achieve multithreaded performance gains.
> writing multithreaded code will get harder because you'll likely need to handle concurrency issues yourself I'd phrase this differently: Writing correct multithreaded code will be just as challenging (or not, depending on the person and their comfort with concurrent and parallel code development) as before, but now you won't be able to get away with sloppy multithreaded code that relied on the GIL to not break.
Earlier quoted context omitted.
If you have a non trivial application, multiprocessing just takes a lot of memory. Every child process that you create duplicates the parent memory. There are some interesting hacks like gc.freeze that exploits the copy on write feature of forks to reduce memory, but ultimately you can just create a few hundred of processes compared to thousands of threads because of memory consumption.
> If you have a non trivial application, multiprocessing just takes a lot of memory. Every child process that you create duplicates the parent memory. Not really, unless you want to alter it. The OS uses copy on write behind the scenes for forked processes, so will use the same memory locations already loaded until/if you modify that. So parent memory isn't really duplicated. As for any new memory allocated by each c…