Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

gh-116167: Allow disabling the GIL

github.com

Post reply on HN
Print view View original

221–230 of 259 posts

Re: gh-116167: Allow disabling the GIL

#221

Post by snovv_crash »

KaiserPro wrote: post #165
ebiester wrote: post #35

Earlier quoted context omitted.

And I guess what I don't understand is why people choose Python for these use cases. I am not in the "Rustify" everything camp, but Go + C, Java + JNI, Rust, and C++ all seem like more suitable solutions.

> but Go + C, Java + JNI, Rust, and C++ all seem like more suitable solutions. apart from go (maybe java) those are all "scary" languages that require a bunch of engineering to get to the point that you can prototype. even then you can normally pybind the bits that are compute bound. If Microsoft had been better back in the say, then c# should have been the goto language of choice. It has the best tradeoff of speed/h…

#pragma omp parallel for

gets you 90% of the potential performance of a full multithreaded producer/consumer setup in C++. C++ isn't as scary as it used to be.

Re: gh-116167: Allow disabling the GIL

#222

Post by nextlevelwizard »

aunderscored wrote: post #166
nextlevelwizard wrote: post #152

Earlier quoted context omitted.

except nothing enforces your types at run time, you can have typi hints all you want and everyone else cn ignore them

Same exists with laws. And in most languages if I want to avoid the type system and hand you a goldfish instead of an int, I can. It just may take more effort. Other language will blow up too if you hand them strange things. Just like python those languages have ways to verify you're not passing goldfish, You just may need more or less effort to use them

Thats a lot of words for: yeah, you are right

Re: gh-116167: Allow disabling the GIL

#223

Post by nextlevelwizard »

bmitc wrote: post #160
nextlevelwizard wrote: post #155

Earlier quoted context omitted.

most software doesnt need multi threading. most times people cry about pythons performance then write trivial shit programs that take milliseconds to run in python as well

Nearly every time I've interactive with Python, its execution speed is absolutely an issue.

Please do give an example.

I see is people crying how python is slow and then use a proper fast programming language to write code that gets executed so few times that even if python was 100x slower it wouldn't matter or the program is so trivial that python's speed definitely isn't an issue.

I have even sometimes seen people stop using a tool when they find out they were written in python - now all of a sudden they are unusably slow. Then they try to justify it by writing some loop in their favourite proper fast language and tell me how fast that tight loop is or they claim that some function is X times faster, but when I actually compile it and run something like hyperfine on it and python version the difference is hardly ever X since there is already so much more over head in a real world.

Re: gh-116167: Allow disabling the GIL

#224

Post by flakes »

progmetaldev wrote: post #196
hn_throwaway_99 wrote: post #144

Earlier quoted context omitted.

Thanks for posting that, I thought it was a great read (as someone who last used C++ probably about 25 years ago...) Given that so many of the criticisms were about C++ being over-complicated, I do worry about languages just becoming more and more difficult over time as everyone wants their pet feature added, but due to backwards-compatibility concerns old/obsolete features are rarely removed. For example, take Java.…

I haven't used Java since v1.5, roughly around 2005. I do use C# quite a bit, and have since v1.1 (I got into .NET through VB.NET at v1.0, as I had just learned VB through schooling). I look back at all the features that have been added over time, and since I have followed along as it has developed, I embrace the changes. They have made my code more concise and easier to read, with less boilerplate. When I think abou…

> I wouldn't be surprised if he's going to be dealing with Java v1.5 instead of the latest features.

Well I certainly hope not… LTS for Java 5 probably hit end of life when he was 2. I’d say most likely the version being installed is Java 17, or 11 if theyre really dated. Java 8 would likely be the absolute minimum, mainly due to the industry being so slow to migrate away from it. Newer programmers are unlikely to ever know what Java felt like before generics and lambdas existed.

Re: gh-116167: Allow disabling the GIL

#225

Post by boxed »

nerpderp82 wrote: post #97
Hugsun wrote: post #26

It will be very exciting to see how much faster they'll be able to make vanilla python. The value proposition is being challenged by the plethora of tools aiming to alleviate those issues. Speed improvers like Mojo, pytorch, triton, numba, taichi come to mind. There are so many different attempts at solving this problem that that the last time I wanted to try one of them, I found myself overwhelmed with options. I ch…

Mojo should be viewed as an attack on the Python ecosystem due to it being a superset. It can consume Python, but it itself is not Python. Taichi is really underrated, it works across all platforms (including Metal), has tons of examples and the code is easy to write. And lastly, it integrates with the ecosystem and doesn't displace it. https://github.com/taichi-dev great demo reel of what Taichi can do, https://www.…

Mojo is not a superset. Not even close. They say they are AIMING for it to be a superset OF THE SYNTAX. This is a subtle yet enormous difference to being a superset of the language itself.

Re: gh-116167: Allow disabling the GIL

#226

Post by bmitc »

bb88 wrote: post #214
rmbyrro wrote: post #134

Earlier quoted context omitted.

The whole purpose of threads is to improve overall speed of execution. Unless you're working with a very small number of threads (single digits), that's a very hard to achieve goal in Python. I wouldn't count this as easy to use. It's easy to program, yes, but not easy to get working with reasonably acceptable performance.

And the python people would just point to multiprocessing...which works pretty well.

Which has its own set of challenges and yet another implementation of queue.

Re: gh-116167: Allow disabling the GIL

#227

Post by aoanla »

shiroiushi wrote: post #215
jakobnissen wrote: post #120

Earlier quoted context omitted.

Julia's JIT compiles code when its first executed, so Julia has a noticable delay from you start the program and until it starts running. This is anywhere from a few hundred milliseconds for small scripts, to tens of seconds or even minutes for large packages.

I wonder why they don't just have an optional pre-compilation, so once you have a version you're happy with and want to run in production, you just have a fully compiled version of the code that you run.

Effectively, it does - one of the things recent releases of Julia have done is to add more precompilation caching on package install. Julia 1.10 feels considerably snappier than 1.0 as a result - that "first time to plot" is now only a couple of seconds thanks to this (and subsequent plots are, of course, much faster than that).

Re: gh-116167: Allow disabling the GIL

#228

Post by imtringued »

CJefferson wrote: post #211
hsbauauvhabzb wrote: post #178

Earlier quoted context omitted.

What packages might depend on the GIL and why would they need it?

Almost all packages depend on the GIL (at least at first). They assume they can "take the GIL", and then go and look at the various Python datastructures you passed them without worrying about them changing as they are being read. The later, when writing out their answer (which might involve editing something they were given), they can assume they are not changing. For example, you could write code which extends the…

This only applies to native extensions.

Re: gh-116167: Allow disabling the GIL

#229

Post by imtringued »

agent281 wrote: post #179
rnmmrnm wrote: post #36

More than seeing it in main, I'm happy for the "python thread slow" meme officially going away now.

I doubt that Python will ditch the meme. The fundamental model of dynamic dispatch using dictionaries on top of a byte code interpreter is pretty slow. I wouldn't expect it to get within 2x of JavaScript.

Ok but given enough cores even python code will run into memory bandwidth problems rather than be bottlenecked by memory latency.

Re: gh-116167: Allow disabling the GIL

#230

Post by imtringued »

mixmastamyk wrote: post #117
sneed_chucker wrote: post #100

Earlier quoted context omitted.

CPython is slow. That's not really something you can dispute. It is a non-optimizing bytecode interpreter and it makes no use of JIT compilation. JavaScript with V8 or any other modern JIT JS engine runs circles around it. Go, Java, and C# are an order of magnitude faster but they have type systems that make optimizing compilation much easier. There's no language-inherent reason why Python can't be at least as fast a…

I've read that it can't even be as fast as JS, because everything is monkey-patchable at runtime. Maybe they can optimize for that when it doesn't happen, but remains to be seen.

You can always use optimistic optimization strategies where you profile the fast path and optimize that. When someone does something slow, you tell them to stop doing it if they want better performance.
Post reply on HN