Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

gh-116167: Allow disabling the GIL

github.com

Post reply on HN
Print view View original

151–160 of 259 posts

Re: gh-116167: Allow disabling the GIL

#151

Post by dimitrios1 »

hn_throwaway_99 wrote: post #144
Kranar wrote: post #126

Earlier quoted context omitted.

While Ken Thompson never used the word attack, he certainly didn't have a positive opinion of the language or of Bjarne Stroustrup either in terms of his technical contributions or his handling of C++ adoption: https://gigamonkeys.wordpress.com/2009/10/16/coders-c-plus-p...

Thanks for posting that, I thought it was a great read (as someone who last used C++ probably about 25 years ago...) Given that so many of the criticisms were about C++ being over-complicated, I do worry about languages just becoming more and more difficult over time as everyone wants their pet feature added, but due to backwards-compatibility concerns old/obsolete features are rarely removed. For example, take Java.…

I was doing primarily go development since it was first released up until a few years ago when the pandemic allowed me the opportunity to move into a full time remote gig doing primarily Java development, so I can answer this as I hadn't done Java at that point for over 10 years, so I felt completely new (what Java I did before that, I was mostly trying to not use Java by using play framework or jruby on rails)

As someone in the boat you mentioned (sort of) the short answer is modern Java development for 90% of tasks is not complicated at all: it's very much like any programming language used in a bizdev/corp environment -- you are mostly using a framework and a bunch of DSLs. Almost everyone uses Intellij and Gradle for IDE and build, and Junit5 or Spock for unit testing. I passed a technical interview mostly on Spring Framework concepts knowing almost nothing about it, nor having ever used it in production by simply just having the documentation open while I was being interviewed so I could look up the answers. Any language that is popular is going to have frameworks with decent documentation that help you be productive quickly, so I just jumped in doing Spring. The java stuff came as needed, or I referenced something like Effective Java (great book), or a Baeldung article. Java world has made some great strides since the 2000's and early 2010s of XML chaos. It took a while, but I feel like it's in a really good spot and getting better.

As an aside, if it hasn't been mentioned to you before, if you like simplicity in a language, but still incredibly productive, you might enjoy Go.

Re: gh-116167: Allow disabling the GIL

#152

Post by nextlevelwizard »

ralphist wrote: post #85
cmeacham98 wrote: post #48

Earlier quoted context omitted.

This seems a bit like saying "JavaScript supports types!" because of typescript.

It's not a separate language, you can just start typing your programs right now.

except nothing enforces your types at run time, you can have typi hints all you want and everyone else cn ignore them

Re: gh-116167: Allow disabling the GIL

#153

Post by vlovich123 »

orf wrote: post #139
vlovich123 wrote: post #110

Earlier quoted context omitted.

I think you would do it on first access - “if new thread, increment atomic & exchange for a new object reference that has the local thread id affinity”. That way you don’t care about whether an object actually has thread affinity or not and you solve the “accessed by many threads” piece. But thanks for answering - I figured complexity was the reason a simpler choice was made to start with.

But this would now make the reference count increment require a conditional? It’s a very hot path, and this would cause a slowdown for single-threaded Python code.

It's already taking a conditional. Take a look at the PEP:

    if (op->ob_tid == _Py_ThreadId())
      op->ob_ref_local = new_local;
    else
      atomic_add(&op->ob_ref_shared, 1
So you're either getting a correct branch prediction or an atomic operation which will dominate the overhead of the branch anyway. All this is saying is in the else branch where you're doing the atomic add, create a new PythonObj instance that has `ob_tid` equal to `_Py_ThreadId`. This presumes that Py_INCREF changes the return type from void to `PythonObj*` and this propagates out so that futher on-thread references use the newer affinity (branch condition is always taken to the non-atomic add instead of the atomic one). It's easier said than done and there may be technical reasons why that's difficult / not possible, but worth exploring eventually so that access by multiple threads of a single object doesn't degrade to taking atomic reference counts constantly.

https://peps.python.org/pep-0703/

Re: gh-116167: Allow disabling the GIL

#154

Post by maest »

fragmede wrote: post #115
tommiegannert wrote: post #50

First I read the news of tranched bread, and now this?! What a time! I was a bit disheartened when the Unladen Swallow project [1] fizzled out. Great to see Python back on the core optimization track. [1] https://en.wikipedia.org/wiki/CPython#Unladen_Swallow

tranched bread?

GP is joking that this is the best thing since sliced (tranched) bread.

Re: gh-116167: Allow disabling the GIL

#155

Post by nextlevelwizard »

bmitc wrote: post #127
scubbo wrote: post #37

Earlier quoted context omitted.

I wish I had your optimism. Thoughtless bandwagon-y "criticism" is extraordinarily persistent.

It isn't thoughtless. I'm working in Python after having come from more designed languages, and concurrency in Python is an absolute nightmare. It feels like using a language from the 60s. An effectively single threaded language in 2024! That's really astonishing.

most software doesnt need multi threading. most times people cry about pythons performance then write trivial shit programs that take milliseconds to run in python as well

Re: gh-116167: Allow disabling the GIL

#157

Post by nick238 »

samatman wrote: post #44
scubbo wrote: post #37

Earlier quoted context omitted.

I wish I had your optimism. Thoughtless bandwagon-y "criticism" is extraordinarily persistent.

There's no need to pretend Python has virtues which it lacks. It's not a fast language. It's fast enough for many purposes, sure, but it isn't fast, and this work is unlikely to change that. Fast er , sure, and that's great.

Maybe it'll shut up "architects" who hack up a toy example in , drop it on a team to add all the actual features, tests, deployment strategy, and maintain, and fly away to swoop and poop on someone else. Gee thanks for your insight; this API serves maybe 1 request a second, tops. Glad we optimized for SPEEEEEED of service over speed of development.

Re: gh-116167: Allow disabling the GIL

#158

Post by IshKebab »

Difwif wrote: post #142
IshKebab wrote: post #108

Earlier quoted context omitted.

Well, technically it still won't be able to use the full power of threads in many situations because (I assume) it doesn't have shared memory. It'll presumably be like Web Workers / isolates, so Go, C++, Rust, Zig, etc. will still have a fundamental advantage for most applications even ignoring Python's inherent slowness. Probably the right design though.

Why would you think it's not shared memory? Maybe I'm wrong here but by default Python's existing threading implementation uses shared memory. AFAIK we're just talking about removing the global interpreter lock. I'm pretty sure the threading library uses system threads. So running without the GIL means actual parallelism across system threads with shared memory access.

Yeah I think you're right actually. Seems like they do per-object locking instead.

Re: gh-116167: Allow disabling the GIL

#159

Post by nick238 »

cmeacham98 wrote: post #48
scubbo wrote: post #39

Earlier quoted context omitted.

Python supports types! https://www.mypy-lang.org/

This seems a bit like saying "JavaScript supports types!" because of typescript.

Python actually has type safety though, as you can't do `'1' + 1` like in JS (not that a linter wouldn't scream at you). If I hear another "I compile so I know it will work, but you can't do that in Python" I'll lose it. Having the compiler not complain that the types match is not effing "testing".

Re: gh-116167: Allow disabling the GIL

#160

Post by bmitc »

nextlevelwizard wrote: post #155
bmitc wrote: post #127

Earlier quoted context omitted.

It isn't thoughtless. I'm working in Python after having come from more designed languages, and concurrency in Python is an absolute nightmare. It feels like using a language from the 60s. An effectively single threaded language in 2024! That's really astonishing.

most software doesnt need multi threading. most times people cry about pythons performance then write trivial shit programs that take milliseconds to run in python as well

Nearly every time I've interactive with Python, its execution speed is absolutely an issue.
Post reply on HN