Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

gh-116167: Allow disabling the GIL

github.com

Post reply on HN
Print view View original

211–220 of 259 posts

Re: gh-116167: Allow disabling the GIL

#211

Post by CJefferson »

hsbauauvhabzb wrote: post #178
nextaccountic wrote: post #140

Earlier quoted context omitted.

If any package depends on the GIL, it will be enabled. Packages won't break

What packages might depend on the GIL and why would they need it?

Almost all packages depend on the GIL (at least at first).

They assume they can "take the GIL", and then go and look at the various Python datastructures you passed them without worrying about them changing as they are being read.

The later, when writing out their answer (which might involve editing something they were given), they can assume they are not changing. For example, you could write code which extends the length of a list to 100, then fill in all the members, not worrying half way through your loop another thread shrinks the list back down.

Re: gh-116167: Allow disabling the GIL

#212

Post by dang »

lbj wrote: post #70
dang wrote: post #57

Earlier quoted context omitted.

" Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something. " " Don't be snarky. " https://news.ycombinator.com/newsguidelines.html

Fair enough, my apologies.

Appreciated!

Re: gh-116167: Allow disabling the GIL

#213

Post by bb88 »

Shog9 wrote: post #182
bb88 wrote: post #99

Earlier quoted context omitted.

Working with threads is a pain regardless of which language you use. Some might say: "Use Go!" Alas: https://songlh.github.io/paper/go-study.pdf After a couple decades of coding, I can say that threading is better if it's tightly controlled, limited to usages of tight parallelism of an algorithm. Where it doesn't work is in a generic worker pool where you need to put mutex locks around everything -- and then prod ran…

This is where Python's GIL bit me: I was more than familiar with how to shoot myself in the foot using threads in other languages, and careful to avoid those traps. Threads spun up only in situations where they had their own work to do and well-defined conditions for how both failure and success would be reported back to the thread that requested it, along with a pool that wouldn't exceed available resources. Like ev…

Not disagreeing. The only case for threading in python is for spinning something to handle IO.

But now with async even that goes away.

Re: gh-116167: Allow disabling the GIL

#214

Post by bb88 »

rmbyrro wrote: post #134
amethyst wrote: post #132

Earlier quoted context omitted.

> Python makes it incredibly tough. I disagree, Python makes it incredibly easy to work with threads in many different ways. It just doesn't make threads faster .

The whole purpose of threads is to improve overall speed of execution. Unless you're working with a very small number of threads (single digits), that's a very hard to achieve goal in Python. I wouldn't count this as easy to use. It's easy to program, yes, but not easy to get working with reasonably acceptable performance.

And the python people would just point to multiprocessing...which works pretty well.

Re: gh-116167: Allow disabling the GIL

#215

Post by shiroiushi »

jakobnissen wrote: post #120
znpy wrote: post #109

Earlier quoted context omitted.

> It is also faster than Python, except for the jit issues. I was intrigued by Julia a while ago, but didn't have time to properly learn it. So just out of curiosity: what's the issues with jit and Julia ?

Julia's JIT compiles code when its first executed, so Julia has a noticable delay from you start the program and until it starts running. This is anywhere from a few hundred milliseconds for small scripts, to tens of seconds or even minutes for large packages.

I wonder why they don't just have an optional pre-compilation, so once you have a version you're happy with and want to run in production, you just have a fully compiled version of the code that you run.

Re: gh-116167: Allow disabling the GIL

#216

Post by kelnos »

richrichie wrote: post #188
Kranar wrote: post #126

Earlier quoted context omitted.

While Ken Thompson never used the word attack, he certainly didn't have a positive opinion of the language or of Bjarne Stroustrup either in terms of his technical contributions or his handling of C++ adoption: https://gigamonkeys.wordpress.com/2009/10/16/coders-c-plus-p...

Thanks for sharing. > perhaps Erik Naggum, scourge of Usenet, was right when he said: “life is too long to know C++ well.” I feel similar about Rust. I have read “the book”. Did a couple of small projects. It is also sort of committee administered language that sucks up tiny features like a giant vacuum cleaner sweeping the streets and changes every hour.

> that sucks up tiny features like a giant vacuum cleaner sweeping the streets and changes every hour.

Can you provide some (preferably recent) examples? My experience has been the opposite. It feels like new features are given a lot of thought and deliberation, and stablizing even something small can take upwards of a year.

I will agree that today's Rust is different from Rust 1.0, but I don't see that as necessarily a bad thing. More that they were able to come to a stable 1.0-ready core fairly early on, and have been adding on the more tricky parts bit by bit since then.

Re: gh-116167: Allow disabling the GIL

#217

Post by Austizzle »

fireattack wrote: post #201
dathery wrote: post #161

Earlier quoted context omitted.

Previously people basically didn't bother to write multithreaded Python at all due to the GIL. Threads were primarily used when you had multiple pieces of work to do which could end up blocked on independent I/O. Which is common and useful of course, but doesn't help with the performance of CPU-bound Python code. Even outside of high-intensity CPU work, this can be useful. A problem lately is that a lot of code is wr…

Would GIT affect how Python runs across multiple Python processes? I'm asking because I encountered a weird phenomenon before. I use a simple Python lib called "schedule" which is to run some tasks periodically (not precise). And I often run a script multiple times (with different arguments) to monitor something say, every 30 seconds. So they're in three separate Python Interpreter processes. What I've noticed is tha…

The GIL is per interpreter/process, so this wouldn't be related as far as I know.

The GIL only really kicks in if you use threads in a single process. Then, the GIL will only let one single thread do actual work at a time, and will trade off which thread gets to do work. The other threads can wait on IO stuff (web requests, the file system, etc) but they can't do number crunching or data processing at the same time.

That's a really interesting observation though, I wonder what _is_ causing your separate processes to sync up?

Re: gh-116167: Allow disabling the GIL

#218

Post by kelnos »

nerpderp82 wrote: post #133
KerrAvon wrote: post #122

Earlier quoted context omitted.

I think "attack" is a bit much; C++ isn't an attack on C.

I didn't mention C++ at all. Was that my argument? This thread is about Python, the GIL. Mojo was brought up as a way to speed up Python code. C++ predates on C in a similar way to how Mojo predates on Python. At least C++ has extern C. https://docs.modular.com/mojo/manual/python/#call-mojo-from-... > As shown above, you can call out to Python modules from Mojo. However, there's currently no way to do the reverse—imp…

I think the person you're replying to is just trying to use an analogous example; you didn't need to bring it up.

Regardless, I think it's a bit alarmist and overly aggressive to assume nefarious intent. Have the developers acted in ways such that this reputation is deserved?

Also, little OT, but it took me unreasonably long to understand that you meant "predates" as the verb form of "predator", not as in "comes before chronologically". The phrase "preys on" may be more clear.

Re: gh-116167: Allow disabling the GIL

#219

Post by oivey »

android42 wrote: post #138
KerrAvon wrote: post #122

Earlier quoted context omitted.

I think "attack" is a bit much; C++ isn't an attack on C.

I wasn't sure whether to agree with this or not, so I finally took a slightly closer look at Mojo just now. This depends on how they license it going forward, and whether they make it open, or use being a superset as a way to capture then trap python users in their ecosystem, and I don't think we have a certain answer which path they'll take yet. The way they let you mix python compatible code with their similar but…

The typing thing is really unnecessary and a step backwards, IMO. Added typing is maybe great for managing a code base, but it isn’t necessary for performance with a compiler that can do type inference.

Re: gh-116167: Allow disabling the GIL

#220

Post by cwalv »

nextlevelwizard wrote: post #155
bmitc wrote: post #127

Earlier quoted context omitted.

It isn't thoughtless. I'm working in Python after having come from more designed languages, and concurrency in Python is an absolute nightmare. It feels like using a language from the 60s. An effectively single threaded language in 2024! That's really astonishing.

most software doesnt need multi threading. most times people cry about pythons performance then write trivial shit programs that take milliseconds to run in python as well

Python being slow, and working to speed python programs up, helped me immensely to build a mental model for what makes programs slow. After learning C in school, when I first learned how python was implemented, I was shocked that it was even usable.
Post reply on HN