Good to hear. The authors are touching on the journey it is to make Cython continue to work. I wonder how hard it'll be to continue to provide bdist packages, or within what timeframe, if at all, Cython can transparently ensure correctness for a no-gil build. Anyone got any insights?
Free-threaded CPython is ready to experiment with
71–80 of 398 posts
Re: Free-threaded CPython is ready to experiment with
#72Earlier quoted context omitted.
Non-blocking HTTP request is the bread and butter use case for asyncio. Most JS projects are doing something like this, and they don't need to manage threads for it. You want to manage your own thread pool for this, or are you going to spawn and kill a thread every time you make a request?
> Non-blocking HTTP request is the bread and butter use case for asyncio And the amount of contorting that has to be done for it in Python would be hilarious if it weren't so sad. > Most JS projects I don't know what JavaScript does, but I do know that Python is not JavaScript. > You want to manage your own thread pool for this... In Python, concurrent futures' ThreadPoolExecutor is actually nice to use and doesn't r…
ThreadPoolExecutor is the most similar thing to asyncio: It hands out promises, and when you call .result(), it's the same as await. JS even made its own promises implicitly compatible with async/await. I'm mentioning what JS does because you're describing a very common JS use case, and Python isn't all that different.
If you have async stuff happening all over the place, what do you use, a global ThreadPoolExecutor? It's not bad, but a bit more cumbersome and probably less efficient. You're running multiple OS threads that are locking, vs a single-threaded event loop. Gets worse the more long-running blocking calls there are.
Also, I was originally asking about free threads. GIL isn't a problem if you're just waiting on I/O. If you want to compute on multiple cores at once, there's multiprocessing, or more likely you're using stuff like numpy that uses C threads anyway.
Re: Free-threaded CPython is ready to experiment with
#73Earlier quoted context omitted.
> Non-blocking HTTP request is the bread and butter use case for asyncio And the amount of contorting that has to be done for it in Python would be hilarious if it weren't so sad. > Most JS projects I don't know what JavaScript does, but I do know that Python is not JavaScript. > You want to manage your own thread pool for this... In Python, concurrent futures' ThreadPoolExecutor is actually nice to use and doesn't r…
I feel you. I know asyncio is "the future", but I usually just want to write a background task, and really hate all the gymnastics I have to do with the color of my functions.
I feel like "asyncio is the future" was invented by the same people who think it's totally normal to switch to a new javascript web framework every 6 months.
Re: Free-threaded CPython is ready to experiment with
#74Earlier quoted context omitted.
I think static typing is a waste of time, but given that you want it, I can see why you wouldn't want to use Python. Its type-checking is more half-baked and cumbersome than other languages, even TS.
Typescript is pretty much the gold standard, it’s amazing how much JavaScript madness you can work around just on the typechecking level. IMHO Python should shamelessly steal as much typescript’s typing as possible. It’s tough since the Microsoft typescript team is apparently amazing at what they do so for now it’s a very fast moving target but some day…
Well the TS tooling is more painful in ways. It's not compatible with some stuff like the NodeJS profiler. Also there's some mess around modules vs "require" syntax that I don't understand fully but TS somehow plays a role.
Re: Free-threaded CPython is ready to experiment with
#75Earlier quoted context omitted.
Right now there is a significant single-threaded performance cost. Somewhere from 30-50%. Part of what my colleague Ken Jin and others are working on is getting back some of that lost performance by applying some optimizations. Expect single-threaded performance to improve for Python 3.14 next year.
To be honest, that seems a lot. Even today a lot of code is single-threaded, and this performance hit will also affect a lot of code running in parallel today. There have been patches to remove the GIL going back to the 90s and Python 1.5 or thereabouts. But the performance impact has always been the show-stopper.
It’s an experimental release in 3.13. Another example: objects that will have deffered reference counts in 3.14 are made immortal in 3.13 to avoid scaling issues from reference count thrashing. This wasn’t originally the plan but deferred reference counting didn’t land in time for 3.13. It will be several years before free-threading becomes the default, at that point there will no longer be any single-threaded performance drop. Of course that assumes everything shakes out as planned, we’ll see.
This post is a call to ask people to “kick the tires”, experiment, and report issues they run into, not announcing that all work is done.
Re: Free-threaded CPython is ready to experiment with
#76Earlier quoted context omitted.
I don't get how this optional static typing works. I had a quick look at [1], and it begins with a note saying that Python's runtime doesn't enforce types, leaving the impression that you need to use third-party tools to do actual type checking. But then it continues just like Python does the check. Consider that I'm not a Python programmer, but the main reason I stay away from it is the lack of a proper type system.…
I think static typing is a waste of time, but given that you want it, I can see why you wouldn't want to use Python. Its type-checking is more half-baked and cumbersome than other languages, even TS.
I used to think like that until I tried.
There are areas where typing is more important: public interfaces. You don't have to make every piece of your program well-typed. But signatures of your public functions / methods matter a lot, and from them types of many internal things can be inferred.
If your code has a well-typed interface, it's pleasant to work with. If interfaces of the libraries you use are well-typed, you have easier time writing your code (that interacts with them). Eventually you type more and more code you write and alter, and keep reaping the benefits.
Re: Free-threaded CPython is ready to experiment with
#77Earlier quoted context omitted.
I think static typing is a waste of time, but given that you want it, I can see why you wouldn't want to use Python. Its type-checking is more half-baked and cumbersome than other languages, even TS.
I used to think like that until I tried. There are areas where typing is more important: public interfaces. You don't have to make every piece of your program well-typed. But signatures of your public functions / methods matter a lot, and from them types of many internal things can be inferred. If your code has a well-typed interface, it's pleasant to work with. If interfaces of the libraries you use are well-typed,…
I shouldn't have said it's a waste of time period, cause every project I work on does have static typing in two very important places: the RPC or web API (OpenAPI, gRPC, whatever it is), and the relational database. But not in the main JS or Py code. That's all I've ever needed.
I did try migrating a NodeJS backend to TS along with a teammate driving that effort. The type-checking never ended up catching any bugs, and the extra time we spent on that stuff could've gone into better testing instead. So it actually made things more dangerous.
Re: Free-threaded CPython is ready to experiment with
#78Earlier quoted context omitted.
> Non-blocking HTTP request is the bread and butter use case for asyncio And the amount of contorting that has to be done for it in Python would be hilarious if it weren't so sad. > Most JS projects I don't know what JavaScript does, but I do know that Python is not JavaScript. > You want to manage your own thread pool for this... In Python, concurrent futures' ThreadPoolExecutor is actually nice to use and doesn't r…
ThreadPoolExecutor is the most similar thing to asyncio: It hands out promises, and when you call .result(), it's the same as await. JS even made its own promises implicitly compatible with async/await. I'm mentioning what JS does because you're describing a very common JS use case, and Python isn't all that different. If you have async stuff happening all over the place, what do you use, a global ThreadPoolExecutor?…
> Python isn't all that different
Again, Python's implementation of asyncio does not allow you to background worker code without explicitly altering that worker code to be aware of asyncio. Threads do. They just don't occupy the same space.
> Also, I was originally asking about free threads...there's multiprocessing
Eh, the obvious reason to not want to use separate processes is a desire for some kind of shared state without the cost or burden of IPC. The fact that you suggested multiprocessing.Pool instead of concurrent_futures.ProcessPoolExecutor and asked about manual pool management feels like it tells me a little bit about where your head is at here wrt Python.
Re: Free-threaded CPython is ready to experiment with
#79Earlier quoted context omitted.
I think static typing is a waste of time, but given that you want it, I can see why you wouldn't want to use Python. Its type-checking is more half-baked and cumbersome than other languages, even TS.
I used to think like that until I tried. There are areas where typing is more important: public interfaces. You don't have to make every piece of your program well-typed. But signatures of your public functions / methods matter a lot, and from them types of many internal things can be inferred. If your code has a well-typed interface, it's pleasant to work with. If interfaces of the libraries you use are well-typed,…
This was the thing that started to bring me around to optional typing as well. It makes the most sense to me as a form of documentation - it's really useful to know what types are expected (and returned) by a Python function!
If that's baked into the code itself, your text editor can show inline information - which saves you from having to go and look at the documentation yourself.
I've started trying to add types to my libraries that expose a public API now. I think it's worth the extra effort just for the documentation benefit it provides.
Re: Free-threaded CPython is ready to experiment with
#80Earlier quoted context omitted.
I guess that depends from your perspective. I'm not a Python developer, but like many people I do want to run Python programs from time to time. I don't really know Rust, or Cargo, but I never have trouble building any Rust program: "cargo build [--release]" is all I need to know. Easy. Even many C programs are actually quite easy: "./configure", "make", and optionally "make install". "./configure" has a nice "--help…
"I don't want a bunch of venvs" That's your problem right there. Virtual environments are the Python ecosystem's solution to the problem of wanting to install different things on the same machine that have different conflicting requirements. If you refuse to use virtual environments and you install more than one separate Python project you're going to run into conflicting requirements and it's going to suck. Have you…
Managing a farm of virtualenvs and mucking about with my PATH doesn't address the user-installable problem at all. And it seems there's a new tool to try every few months that really will fix all problems this time.
And maybe if you're a Python developer working on the code every day that's all brilliant. But most people aren't Python developers, and I just want to try that "Show HN" project or whatnot.
Give me a single command I can run. Always. For any project. And that always works. If you don't have that then your build system needs work.