Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's Remove the Global Interpreter Lock

morepypy.blogspot.com

Post reply on HN
Print view View original

311–320 of 326 posts

Re: Let's Remove the Global Interpreter Lock

#311

Post by alfanerd »

zepolen wrote: post #298
alfanerd wrote: post #294

I just made a quick test: CPython-3.6.1 vs. Jython-2.7.0 (May 2015) I ran Larry Hastings' Gilectomy testprogram x.py: fib(40) on 8 threads HW: MacBook Pro, 2015, 8 (4+4) cores, 1Gb RAM Jython ran the program 8 times faster, utilising all 8 cores >95%. Python ran on 1-2 cores less than 60% utilisation. (Pretty sure Jython will run 16 times faster on 16 cores) It's 2017, why this is acceptable to GvR and the Python com…

Could you post the code for that fib program I couldn't find it anywhere.

It is in the Gilectomy branch of Larry Hastings github project. (https://github.com/larryhastings/gilectomy)

I also pasted the fib test to pastebin: https://pastebin.com/Ryyb2K7V

Re: Let's Remove the Global Interpreter Lock

#312

Post by shwouchk »

myusernameisok wrote: post #151
shwouchk wrote: post #72

Earlier quoted context omitted.

GIL does not help thread safety in application code (and external libraries), just in the VM.

I'm fairly certain you're incorrect. With the GIL you don't have to lock shared memory because the assumption is that only one thread will be running at a time. For example shared data structures won't be changed while being being read/written to by multiple threads, because only one thread is actually running.

You are entirely mistaken, unless all you care about are the basic built in dict/list and some of the other built in data structures AND each thread only stores OR reads data (i.e. never reads and then stores it again), from a SINGLE container (you never care about consistent state between two different objects).

In my experience this is almost never the case. Moreover, this type of synchronization is trivial to accomplish with relatively little performance sacrifice.

What is much more complicated is getting more complex logic work correctly and performantly when you are interacting with multiple different data structures from something more than a saturated loop.

Re: Let's Remove the Global Interpreter Lock

#313

Post by yaroslavvb »

goerz wrote: post #306
throwaway613834 wrote: post #254

Earlier quoted context omitted.

> Even with BLAS enabled, NumPy has almost zero intrinsic parallelism, np.dot() being the notable exception which releases the GIL and uses multicore by itself. Is there any sort of list (comprehensive or otherwise) that denotes which NumPy functions are parallelism-friendly? I mean this whether it's in terms of releasing the GIL, in terms of SIMD support, or in terms of being multi-core.

Is there a way to disable this? In an HPC environment, I don't want routines going multi-core without my explicit permission, under any circumstances. I will already have manually set up the parallelization to be at the highest logical level. If using Python, that usually means I have planned out the number of processes to be equal to the number of cores. If each process then starts doing its own multicore calculatio…

Underlying implementations often have a way to disable parallelism, ie, OMP_NUM_THREADS=1 or MKL_NUM_THREADS=1

Re: Let's Remove the Global Interpreter Lock

#314

Post by sqeaky »

kllrnohj wrote: post #148
sqeaky wrote: post #123

Earlier quoted context omitted.

You are right about the majority of what you said, but I am pedantically picking on one point. CPU cores are getting faster, but they aren't doing it with clock speed, they are dispatching more instructions per cycle or otherwise making the work faster.

IPC gains per generation are vanishingly tiny, if they exist at all. Skylake -> Kaby Lake, for example, had no IPC improvements at all. A very small clock bump to the various tiers was it. Even if you look over a large generation gap there's only a ~20% IPC improvement going from an i7-2600K to an i7-7700K ( https://www.hardocp.com/article/2017/01/13/kaby_lake_7700k_v... ) 6 years & a shrink from 32nm to 14nm and all…

My compiles have gotten more than 20% faster so something is making my newere machines faster than my older machines.

That it is not 10x as fast I blame on AMD for not being as competitive as they could have been.

Re: Let's Remove the Global Interpreter Lock

#315

Post by titanix88 »

neolefty wrote: post #176
omarforgotpwd wrote: post #142

Earlier quoted context omitted.

Motivation for removing the GIL is basically that when people hear about it they go "hmmm that doesn't sound good". Obviously many applications have been written in GIL languages and there aren't really many practical problems that can't be overcome easily.

I think it may be some Stockholm Syndrome -- people have worked very hard to get around the GIL, and they've come to expect its limitations and respect those solutions. But I've never heard of someone asking for a GIL to be added to the JVM.

This! Try to implement a controlled task scheduler using multiprocessing and sooner or later you are going to hit some unexpected behavior, like - multiprocessing.Queue belly-upping for no reason, UNIX signals propagating throughout the process chain and killing them left and right, hitting some data/object which are not serializable etc. Getting multiprocessing to work right takes a LOT of careful efforts, which breaks the whole promise python.

I've since moved to clojure, which is a language designed with concurrency from ground up. Look at clojure's `atom` - it's basically what every beginning programmer expects from globally shared variables, minus the gymnastics of handling race conditions on your own.

Also, `core.async` is such a beautiful thing to work with for writing schedulers. Compared to this, python's asyncio is an unfunny joke.

I don't think python's GIL can be removed with ad-hoc locking. Nothing sort of complete re-implementation will do.

Re: Let's Remove the Global Interpreter Lock

#316

Post by ogrisel »

detroitcoder wrote: post #210
eslaught wrote: post #161

The comments here are missing a massive use case: shared memory. Shared memory isn't just about programmer convenience. It's about using a machine's memory resources more effectively. Yes, shared memory is available in multi-processing, but it doesn't necessarily interact well with existing codes. I've been working on adding Python support to Legion [1], a task-based runtime system for HPC. Legion wants to manage sha…

^This. It is a very common usecase for applications I work with to create a very large in memory read-only pd dataframe and then put a flask interface to operations on that dataframe using gunicorn and expose as an API. If I use async workers, the dataframe operations are bound by GIL restraints. If I use sync workers, each process needs a copy of the pd dataframe which the server cannot handle (I have never seen pre…

The pickling implementation of joblib has support for memory mapping numpy arrays nested in arbitrary data structures such as pandas dataframes.

Save the dataframe in a folder that can be accessed by the gunicorn worker:

    import joblib
    joblib.dump(df, '/folder/shared_data.pkl')
Then in the code run by the flask / gunicorn workers themselves:

    import joblib
    shared_df = joblib.load('/folder/shared_data.pkl', mmap_mode='r')
    # use the shared_df as usual (inplace modifications are not
    # authorized)
Some pandas function can have issues with read-only buffer though: https://github.com/pandas-dev/pandas/issues/17192 (caused by a currently unsolved bug / limitation of Cython) but it can work for your use case.

Re: Let's Remove the Global Interpreter Lock

#317

Post by sevensor »

btilly wrote: post #277
sevensor wrote: post #266

Earlier quoted context omitted.

Thanks for the link! Might be worth going down that path.

Good luck. Another benefit of this strategy is that you optimize that data structure using techniques that aren't available in higher languages. So, for instance, small trees can be set up to have all of the nodes of the tree very close together, improving the odds of a cache hit. You can switch from lots of small strings to having integers that index a lookup table of strings for display only. The amount of work to…

Thanks! I've already partly rewritten it in C once, but I misunderstood the access pattern and I ended up having a lot of cache misses. The speedup was measurable, but disappointing, and the prospect of doing another rewrite had put me off. I hadn't put two and two together about this being an effective way to share memory under multiprocessing until reading this thread, so it's worth revisiting now.

Re: Let's Remove the Global Interpreter Lock

#318

Post by detroitcoder »

zeptomu wrote: post #259
detroitcoder wrote: post #210

Earlier quoted context omitted.

^This. It is a very common usecase for applications I work with to create a very large in memory read-only pd dataframe and then put a flask interface to operations on that dataframe using gunicorn and expose as an API. If I use async workers, the dataframe operations are bound by GIL restraints. If I use sync workers, each process needs a copy of the pd dataframe which the server cannot handle (I have never seen pre…

> create a very large in memory read-only pd dataframe and then put a flask interface to operations on that dataframe using gunicorn and expose as an API. [...] May I ask what you consider large memory - MByte, GByte, TByte? The simplest solution is to store it as a blob on a SSD, and read it via simple file IO or put it into a DB. But I assume this was too slow, so it would be interesting to go into more details. In…

Lets say there are a couple dataframes that need a matrix multiply that take up about 10gb on a 32gb host. I want to parameterize these manipulations and expose over http. I can only afford to cache 3 sets of them, which means that I can perform 3 concurrent requests. I would like to provide more concurrency than this without reading from disk or storing the data out of process in a separate service which adds complexity.

Re: Let's Remove the Global Interpreter Lock

#319

Post by detroitcoder »

ogrisel wrote: post #316
detroitcoder wrote: post #210

Earlier quoted context omitted.

^This. It is a very common usecase for applications I work with to create a very large in memory read-only pd dataframe and then put a flask interface to operations on that dataframe using gunicorn and expose as an API. If I use async workers, the dataframe operations are bound by GIL restraints. If I use sync workers, each process needs a copy of the pd dataframe which the server cannot handle (I have never seen pre…

The pickling implementation of joblib has support for memory mapping numpy arrays nested in arbitrary data structures such as pandas dataframes. Save the dataframe in a folder that can be accessed by the gunicorn worker: import joblib joblib.dump(df, '/folder/shared_data.pkl') Then in the code run by the flask / gunicorn workers themselves: import joblib shared_df = joblib.load('/folder/shared_data.pkl', mmap_mode='r…

This looks very interesting. I am reading the docs https://pythonhosted.org/joblib/parallel.html#manual-managem... and it looks like it would help a lot (possibly solve the issue). Do you have any experience using this in production?

Re: Let's Remove the Global Interpreter Lock

#320

Post by detroitcoder »

detroitcoder wrote: post #319
ogrisel wrote: post #316

Earlier quoted context omitted.

The pickling implementation of joblib has support for memory mapping numpy arrays nested in arbitrary data structures such as pandas dataframes. Save the dataframe in a folder that can be accessed by the gunicorn worker: import joblib joblib.dump(df, '/folder/shared_data.pkl') Then in the code run by the flask / gunicorn workers themselves: import joblib shared_df = joblib.load('/folder/shared_data.pkl', mmap_mode='r…

This looks very interesting. I am reading the docs https://pythonhosted.org/joblib/parallel.html#manual-managem... and it looks like it would help a lot (possibly solve the issue). Do you have any experience using this in production?

DAMN. I just did a basic test and it kinnda just worked?!? I created a test dataframe of 100M rows X 10 cols which took up ~2.3G and then used joblib.dump within the on_starting hook which is run when the gunicorn master starts up. Then loaded that df in with joblib.load within the worker and the total memory consumption was practically flat. Then I bumped up the number of workers to 20 and still flat. That is actually amazing. Coolest thing I have seen in months for how easy it is. Now I have to test out if the analytics actually work and a deep dive into the mechanics of mem-mapping.
Post reply on HN