Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

gh-116167: Allow disabling the GIL

github.com

Post reply on HN
Print view View original

231–240 of 259 posts

Re: gh-116167: Allow disabling the GIL

#231

Post by est »

nerpderp82 wrote: post #97
Hugsun wrote: post #26

It will be very exciting to see how much faster they'll be able to make vanilla python. The value proposition is being challenged by the plethora of tools aiming to alleviate those issues. Speed improvers like Mojo, pytorch, triton, numba, taichi come to mind. There are so many different attempts at solving this problem that that the last time I wanted to try one of them, I found myself overwhelmed with options. I ch…

Mojo should be viewed as an attack on the Python ecosystem due to it being a superset. It can consume Python, but it itself is not Python. Taichi is really underrated, it works across all platforms (including Metal), has tons of examples and the code is easy to write. And lastly, it integrates with the ecosystem and doesn't displace it. https://github.com/taichi-dev great demo reel of what Taichi can do, https://www.…

> Mojo should be viewed as an attack on the Python ecosystem due to it being a superset

A superset of pure .py code, not the numpy, cython, ctypes and stuff.

But once you get "superset" of CPython's C bindings, congratulations, you get GIL.

Re: gh-116167: Allow disabling the GIL

#232

Post by thisismyswamp »

AdamJacobMuller wrote: post #210
purpleidea wrote: post #200

It's neat that this will eventually improve some python code, but at the end of the day, it's still a badly typed language, which will still be slower and less safe than more modern languages. Learn golang or rust instead. Impressive that they are managing this though!

> Learn golang or rust instead. Bad take. Learn golang and rust and python. You should use the language which is suited to the task, sometimes that's golang sometimes that's python and sometimes it's rust. It's impressive that the python team as a whole continues to improve in such big ways after more than 30 years of development. It's more impressive that the python team managed to navigate 2to3 and come out stronge…

> Learn golang

Having to "if err != nil" every single function call is a big put off - imagine having to "try catch" everything in a language like C#!

Re: gh-116167: Allow disabling the GIL

#233

Post by ralphist »

nextlevelwizard wrote: post #152
ralphist wrote: post #85

Earlier quoted context omitted.

It's not a separate language, you can just start typing your programs right now.

except nothing enforces your types at run time, you can have typi hints all you want and everyone else cn ignore them

Everyone else can ignore every coding guideline your team sets up, that’s why code reviews and managers exist.

Re: gh-116167: Allow disabling the GIL

#234

Post by neonsunset »

AdamJacobMuller wrote: post #210
purpleidea wrote: post #200

It's neat that this will eventually improve some python code, but at the end of the day, it's still a badly typed language, which will still be slower and less safe than more modern languages. Learn golang or rust instead. Impressive that they are managing this though!

> Learn golang or rust instead. Bad take. Learn golang and rust and python. You should use the language which is suited to the task, sometimes that's golang sometimes that's python and sometimes it's rust. It's impressive that the python team as a whole continues to improve in such big ways after more than 30 years of development. It's more impressive that the python team managed to navigate 2to3 and come out stronge…

Python is a bad programming language, but a great scripting one. You use it when you either have to do scripting or need a framework that is only offered in Python (the entirety of ML domain). Otherwise, the only reason to pick it is if you have no choice, you don't want to do a rewrite or you are being held at a gun point.

Instead, pick C#/F#, Kotlin/Clojure or Rust depending on the use case.

Re: gh-116167: Allow disabling the GIL

#235

Post by agacera »

behnamoh wrote: post #25

I've been programming in Python for over 6 years now and every week I learn something new. But recently I've been thinking about moving to a more capable language with proper concurrency for backend API requests (FastAPI sucked). I also want types, so Elixir is not in the picture. I dabbled in Rust a bit. Although I was able to get the hang of things and build a CLI tool pretty quickly, I'm worried I'll have to deal…

DevX in Rust for backend development is quite good nowadays. Things are WAY easier than it was a couple of years ago.

I migrated a couple of projects from Java, TS and Go to Rust and honesly, I couldn't be happier.

Re: gh-116167: Allow disabling the GIL

#236

Post by pas »

fragmede wrote: post #163
KMnO4 wrote: post #123

Earlier quoted context omitted.

I could be wrong, but I think it’s a clever alternative to the expression “best thing since sliced bread”.

ahahah. I was thinking that it was a new python library or something that I hadn't heard of and was coming up short with Google. "tranced bread" is a fun name for some sort of library that breaks up files into pieces for better resilience for sending, like over BitTorrent.

tranced bread sounds like what you go home with after a goa festival

Re: gh-116167: Allow disabling the GIL

#237

Post by sneed_chucker »

cozzyd wrote: post #205
sneed_chucker wrote: post #100

Earlier quoted context omitted.

CPython is slow. That's not really something you can dispute. It is a non-optimizing bytecode interpreter and it makes no use of JIT compilation. JavaScript with V8 or any other modern JIT JS engine runs circles around it. Go, Java, and C# are an order of magnitude faster but they have type systems that make optimizing compilation much easier. There's no language-inherent reason why Python can't be at least as fast a…

JavaScript doesn't have to contend with a plethora of native extensions (which, to be fair, are generally a workaround for python slowness).

JavaScript, at least on the Node.JS side, make plenty use of native extensions written in C++ https://nodejs.org/api/addons.html

In any case, that should be irrelevant to getting a reasonably performant JIT running. Lots of AOT and JIT compiled languages have robust FFI functionality.

The native extensions are more relevant when we talk about removing the GIL, since lots of Python code may call into non thread safe C extension code.

Re: gh-116167: Allow disabling the GIL

#238

Post by fulafel »

fulafel wrote: post #208
colatkinson wrote: post #162

Earlier quoted context omitted.

Adding on to the other comment, multiprocessing is also kinda broken on Linux/Mac. 1. Because global objects are refcounted, CoW effectively isn't a thing on Linux. They did add a way to avoid this [0], but you have to manually call it once your main imports are done. 2. On Mac, turns out a lot of the system libs aren't actually fork-safe [1]. Since these get imported inadvertently all the time, Python on Mac actuall…

Re (1), are there publicly documented cases with numbers on observed slowdowns with it? I see this mentioned from time to time, but intuitively you'd think this wouldn't pose a big slowdown since the system builtin objects would have been allocated at the same time (startup) and densely located on smaller nr of pages. I guess if you have a lot of global state in your app it could be more significant. Would also be in…

Replying to my self: it seems one poster case was Instagram and their very large Django app: https://bugs.python.org/issue40255#msg366835

Re: gh-116167: Allow disabling the GIL

#239

Post by swtaarrs »

dvas wrote: post #10

Extra links for the no gil work for anyone else curious about this [0], [1]. [0] Multithreaded Python without the GIL https://docs.google.com/document/d/18CXhDb1ygxg-YXNBJNzfzZsD... [1] Github repo https://github.com/colesbury/nogil

Those links are both fairly old. See PEP 703 [0] and Sam’s nogil-3.12 repo [1] for more current versions.

[0] https://peps.python.org/pep-0703/

[1] https://github.com/colesbury/nogil-3.12

Re: gh-116167: Allow disabling the GIL

#240

Post by its-summertime »

Intralexical wrote: post #187
its-summertime wrote: post #169

Earlier quoted context omitted.

Cython is also a superset, is Cython also guilty of such crimes?

Cython is dependent on CPython. Cython outputs Python extension modules, which can only be used when imported into a standard CPython environment, and which interoperate cleanly with the rest of the Python ecosystem. Mojo explicitly does the opposite, allowing Mojo to use Python but requiring Mojo to be in control, while making it hard/impossible for code written in Mojo to benefit code written in Python: > Our long-…

Cython can also compile to executables which are not to be used with python.

Same for Nuitka, and a bunch of transpilers

Post reply on HN