Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Free-threaded CPython is ready to experiment with

labs.quansight.org

Post reply on HN
Print view View original

131–140 of 398 posts

Re: Free-threaded CPython is ready to experiment with

#131

Post by dagenix »

stavros wrote: post #118
dagenix wrote: post #97

Earlier quoted context omitted.

As I understand it, if your code would have race conditions with free threaded python, than it probably already has them.

Not when there's a global interpreter lock.

Are you writing an extension or Python code?

If you are writing Python code, the GIL can already be dropped at pretty much any point and there isn't much way of controlling when. Iirc, this includes in the middle of things like +=. There are some operations that Python defines as atomic, but, as I recall, there aren't all that many.

In what way is the GIL preventing races for your use case?

Re: Free-threaded CPython is ready to experiment with

#132

Post by zitterbewegung »

nine_k wrote: post #21
zitterbewegung wrote: post #17

Earlier quoted context omitted.

Optional static typing is just like a comment (real term is annotation) of the input variable(s) and return variable(s). No optimization is performed. Using a tool such as mypy that kicks off on a CI/CD process technically enforces types but they are ignored by the interpreter unless you make a syntax error.

A language server in your IDE kicks in much earlier, and is even more helpful.

I haven't used an IDE that has that but it is still just giving you a hint that there is an error and the interpreter is not throwing an error which was my point.

Re: Free-threaded CPython is ready to experiment with

#133

Post by stavros »

dagenix wrote: post #131
stavros wrote: post #118

Earlier quoted context omitted.

Not when there's a global interpreter lock.

Are you writing an extension or Python code? If you are writing Python code, the GIL can already be dropped at pretty much any point and there isn't much way of controlling when. Iirc, this includes in the middle of things like +=. There are some operations that Python defines as atomic, but, as I recall, there aren't all that many. In what way is the GIL preventing races for your use case?

I mean that, if the GIL didn't prevent races, it would be trivially removable. Races that are already there in people's Python code have probably been debugged (or at least they are tolerated), so there are some races that will happen when the GIL is removed, and they will be a surprise.

Re: Free-threaded CPython is ready to experiment with

#135

Post by woodruffw »

andmkl wrote: post #122
DanielVZ wrote: post #55

Earlier quoted context omitted.

Usually performance critical code is written in cpp, fortran, etc, and then wrapped in libraries for Python. Python still has a use case for glue code.

Yes, but then extensions can already release the GIL and use the simple and industrial strength std::thread, which is orders of magnitude easier to debug.

Concurrent operations exist at all levels of the software stack. Just because native extensions might want to release the GIL and use OS threads doesn't mean pure Python can't also want (or need) that.

(And as a side note: I have never, in around a decade of writing C++, heard std::thread described as "easy to debug.")

Re: Free-threaded CPython is ready to experiment with

#137

Post by pansa2 »

PEP703 explains that with the GIL removed, operations on lists such as `append` remain thread-safe because of the addition of per-list locks.

What about simple operations like incrementing an integer? IIRC this is currently thread-safe because the GIL guarantees each bytecode instruction is executed atomically.

Re: Free-threaded CPython is ready to experiment with

#138

Post by ipsod »

andmkl wrote: post #122
DanielVZ wrote: post #55

Earlier quoted context omitted.

Usually performance critical code is written in cpp, fortran, etc, and then wrapped in libraries for Python. Python still has a use case for glue code.

Yes, but then extensions can already release the GIL and use the simple and industrial strength std::thread, which is orders of magnitude easier to debug.

Really? Cool.

I expected that dropping down to C/C++ would be a large jump in difficulty and quantity of code, but I've found it isn't, and the dev experience isn't entirely worse, as, for example, in-editor code-intelligence is rock solid and very fast in every corner of my code and the libraries I'm using.

If anyone could benefit from speeding up some Python code, I'd highly recommend installing cppyy and giving it a try.

Re: Free-threaded CPython is ready to experiment with

#139

Post by woodruffw »

adgjlsfhk1 wrote: post #93
__MatrixMan__ wrote: post #83

Earlier quoted context omitted.

Python's dependency management sucks because they're audacious enough to attempt packaging non-python dependencies. People always bring Maven up as a system that got it right, but Maven only does JVM things. I think the real solution here is to just only use python dependency management for python things and to use something like nix for everything else.

Julia's package manager (for one) works great and can manage non Julia packages. the problem with python's system is that rejecting semver makes writing a package manager basically impossible since there is no way to automatically resolve packages.

Could you clarify what you mean? pip and every other Python package installer is absolutely doing automatic package resolution, and the standard (PEP 440) dependency operators include a compatible version operator (~=) that's predicated on SemVer-style version behavior.

Re: Free-threaded CPython is ready to experiment with

#140

Post by kortex »

GTP wrote: post #34
nine_k wrote: post #20

Earlier quoted context omitted.

At MPOW most Python code is well-type-hinted, and mypy and pyright are very helpful at finding issues, and also for stuff like code completion and navigation, e.g. "go to the definition of the type of this variable". Works pretty efficiently. BTW, Typescript also does not enforce types at runtime. Heck, C++ does not enforce types at runtime either. It does not mean that their static typing systems don't help during a…

> BTW, Typescript also does not enforce types at runtime. Heck, C++ does not enforce types at runtime either. It does not mean that their static typing systems don't help during at development time. Speaking of C here as I don't have web development experience. The static type system does help, but in this case, it's the compiler doing the check at compile time to spare you many surprises at runtime. And it's part of…

https://github.com/python/mypy

> Python itself doesn't do that

The type syntax is python. MyPy is part of Python. It's maintained by the python foundation. Mypy is not part of CPython because modularity is good, the same way that ANSI C doesn't compile anything, that's what gcc, clang, etc are for.

Mojo is literally exactly the same way, the types are optional, and the tooling handles type checking and compilation.

Post reply on HN