Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's Remove the Global Interpreter Lock

morepypy.blogspot.com

Post reply on HN
Print view View original

71–80 of 326 posts

Re: Let's Remove the Global Interpreter Lock

#71

Post by DougN7 »

jrs95 wrote: post #59
seunosewa wrote: post #53

The ideal solution is for someone to design a new programming language that is as similar to Python as possible without requiring a global lock. Rarely used features that make it hard to parallelize Python would be dropped. STM might be built into the language instead of being hacked into one implementation, etc.

Perl 6 doesn't have a GIL, and already has a sane concurrency model, but the lack of libraries and community interest seems to make that pretty much a non-starter.

Well, that and Perl has to be one of the most unreadable languages out there ;)

Re: Let's Remove the Global Interpreter Lock

#72

Post by shwouchk »

myusernameisok wrote: post #8
taf2 wrote: post #4

Earlier quoted context omitted.

If I understand correctly the issue in Ruby is the existing C extensions that have been written to assume the lock exists...

It's the same issue with Python. AFAIK there are a number of Python libraries that are not thread-safe, and the GIL prevents them from being an issue.

GIL does not help thread safety in application code (and external libraries), just in the VM.

Re: Let's Remove the Global Interpreter Lock

#73

Post by [deleted] »

adambyrtek wrote: post #61
seunosewa wrote: post #53

The ideal solution is for someone to design a new programming language that is as similar to Python as possible without requiring a global lock. Rarely used features that make it hard to parallelize Python would be dropped. STM might be built into the language instead of being hacked into one implementation, etc.

Fork with no backward compatibility is hardly "the ideal solution". Healthy ecosystem is crucial for sustainability of any programming language.

[deleted]

Re: Let's Remove the Global Interpreter Lock

#74

Post by mrsteveman1 »

seunosewa wrote: post #53

The ideal solution is for someone to design a new programming language that is as similar to Python as possible without requiring a global lock. Rarely used features that make it hard to parallelize Python would be dropped. STM might be built into the language instead of being hacked into one implementation, etc.

> language that is as similar to Python as possible without requiring a global lock

Something like Pony[0] or Nim[1]? I'm not very familiar with either one, but Nim says it is inspired by Python, and on the surface Pony appears to be as well.

[0] https://bluishcoder.co.nz/2015/11/04/a-quick-look-at-pony.ht...

[1] https://nim-lang.org/features.html

Re: Let's Remove the Global Interpreter Lock

#75

Post by sandGorgon »

dec0dedab0de wrote: post #12

Could someone who really wants to get rid of the GIL explain the appeal? As far as I understand, the only time it would be useful is when you have an application that is 1. Big enough to need concurrency 2. Not big enough to require multiple boxes. 3. Running in a situation that can not spare the resources for multiprocessing. 4. You want to share memory instead of designing your workflow to handle messages or workin…

+1 on this - what is more important for me is some kind of Numba LLVM jit to automatically optimize hotspots : kind of like the JVM hotspot compiler.

Numba already does some of this.

Additionally, I cannot help but wonder if the answer to these problems has been the JVM all along. Especially with JVM 9 and the Truffle framework - https://github.com/securesystemslab/zippy

Re: Let's Remove the Global Interpreter Lock

#76

Post by darpa_escapee »

monkmartinez wrote: post #50
devwastaken wrote: post #13

This would be great if it means we can run the C portions of Python in threads without performance hits. I recently started a little project that is a cross-platform GUI for batch bzip2 compression, and Python did it quite well with its built-in bzip2 module. But, once I tried to do it parallel, the performance impacts of GIL were obvious. Yes, you can work around that with multi-process, but I'd rather not be spammi…

> I'd rather not be spamming the running processes list and have to actually handle seperate processes that should be threads. I may be a bit naive asking this... but why would you care that much? Looking at activity monitor on my Mac, I count 14 Google Chrome Helper Process instances each spawning upwards of 13 threads. Adobe does something similar, as do several other programs/applications on my machine. Yet, my ma…

There's more overhead communicating between processes than there would be if threads were just modifying shared state.

Re: Let's Remove the Global Interpreter Lock

#77

Post by brian_herman »

sandGorgon wrote: post #70

This is a PERFECT usecase for Kickstarter. It makes me sad that this is a blog post that made it number 1 on HN with vast readership with open pursestrings.. yet there is not a campaign fundraising link. Use Kickstarter or Plasso to sell a pypy pro license - its so much easier for companies to pay invoices than to donate. If nothing else, I would pay for an official conda pypy package which works seamlessly with pand…

They can raise funds on their own for example: $67126 of $105000 (63.9%) for py3k in pypy

And for STM in pypy: 2nd call: $59080 of $80000 (73.9%)

Re: Let's Remove the Global Interpreter Lock

#78

Post by nine_k »

seunosewa wrote: post #53

The ideal solution is for someone to design a new programming language that is as similar to Python as possible without requiring a global lock. Rarely used features that make it hard to parallelize Python would be dropped. STM might be built into the language instead of being hacked into one implementation, etc.

If you want other multiplatform, open-source, highly parallel languages with nice syntax and quick turnaround, we already have a few, like Elixir, Racket, or, well, even ES6.

Much of the Python's appeal is in its huge, colossal, powerful ecosystem, with modules for everything, and things like numpy or tensorflow using it as the high-level interface language. Not breaking this is probably more important for success than efficient in-process data sharing. (Yes, process pools, queues, and a shared DB cover most of my cases.)

Re: Let's Remove the Global Interpreter Lock

#79

Post by rguillebert »

andruby wrote: post #62
wiremine wrote: post #58

> We estimate a total cost of $50k... Just looking at it from a financial perspective, having a great Python interpreter that doesn't have a GIL seems like a no brainer for $50,000, and it creates another reason why people should take a look at PyPy. Side note: if you haven't looked at PyPy, check it out, along with RPython https://rpython.readthedocs.io/en/latest/

How can they estimate this? What about all the libraries that might not be compatible with the solution PyPy comes up with? This feels like a number that might in the end blow up to 10x the original estimate.

It's not the PyPy developers' job to make every Python library threadsafe, people writing libraries will have to make their code threadsafe, like in every other language.

Re: Let's Remove the Global Interpreter Lock

#80

Post by ryanjl »

seunosewa wrote: post #53

The ideal solution is for someone to design a new programming language that is as similar to Python as possible without requiring a global lock. Rarely used features that make it hard to parallelize Python would be dropped. STM might be built into the language instead of being hacked into one implementation, etc.

Jython and Iron Python lack the GIL. It's just an implementation detail of the underlying VM. There's nothing in the language itself which requires a GIL.
Post reply on HN