Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

State of Python 3.13 performance: Free-threading

codspeed.io

Post reply on HN
Print view View original

21–30 of 193 posts

Re: State of Python 3.13 performance: Free-threading

#21

Post by eigenspace »

dagmx wrote: post #18
eigenspace wrote: post #15

Earlier quoted context omitted.

Python's community was significantly smaller and less flushed with cash during the 2 to 3 transition. Since then there has been numerous 3.x releases that were breaking and people seem to have been sucking it up and dealing with it quietly so far. The main thing is that unlike the 2 to 3 transition, they're not breaking syntax (for the most part?), which everyone experiences and has an opinion on, they're breaking ra…

I disagree with this entire comment. The Python community consisted of tons of developers including very wealthy companies. At what point in the last few years would you even say they became “rich enough” to do the migration? Because people are STILL talking about trying to fork 2.7 into a 2.8. I also disagree with your assertion that 3.x releases have significant breaking changes. Could you point to any specific maj…

Fair enough. You may be totally right here, as I mentioned I don't use Python much at all since like 2017 and haven't paid it much attention in a while. I retract my comment.

Regarding breakage in 3.x, all I know is that I recall several times where I did a linux system update (rolling release), and that updated my Python to a newly released version which broke various things in my system. I'm pretty sure one of these was v3.10, but I forget which others caused me problems which I could only solve by pinning Python to an older release.

It's entirely possible though that no actual APIs were broken and that this was just accidentaly bugs in the release, or the packages were being naughty and relying on internals they shouldn't have relied on or something else.

Re: State of Python 3.13 performance: Free-threading

#22

Post by Capricorn2481 »

eigenspace wrote: post #3

I don't really have a dog in this race as I don't use Python much, but this sort of thing always seemed to be of questionable utility to me. Python is never really going to be 'fast' no matter what is done to it because its semantics make most important optimizations impossible, so high performance "python" is actually going to always rely on restricted subsets of the language that don't actually match language's "re…

I don't really get this. They have already made Python faster in the past while maintaining the same semantics. Seems like a good goal to me.

Re: State of Python 3.13 performance: Free-threading

#23

Post by dagmx »

eigenspace wrote: post #21
dagmx wrote: post #18

Earlier quoted context omitted.

I disagree with this entire comment. The Python community consisted of tons of developers including very wealthy companies. At what point in the last few years would you even say they became “rich enough” to do the migration? Because people are STILL talking about trying to fork 2.7 into a 2.8. I also disagree with your assertion that 3.x releases have significant breaking changes. Could you point to any specific maj…

Fair enough. You may be totally right here, as I mentioned I don't use Python much at all since like 2017 and haven't paid it much attention in a while. I retract my comment. Regarding breakage in 3.x, all I know is that I recall several times where I did a linux system update (rolling release), and that updated my Python to a newly released version which broke various things in my system. I'm pretty sure one of thes…

To your last point: it’s neither the language nor the packages but rather it’s the ABI.

Python isn’t fully ABI stable (though it’s improved greatly) so you can’t just intermix compiled dependencies between different versions of Python.

This is true for many packages in your distro as well.

Re: State of Python 3.13 performance: Free-threading

#24

Post by andai »

eigenspace wrote: post #3

I don't really have a dog in this race as I don't use Python much, but this sort of thing always seemed to be of questionable utility to me. Python is never really going to be 'fast' no matter what is done to it because its semantics make most important optimizations impossible, so high performance "python" is actually going to always rely on restricted subsets of the language that don't actually match language's "re…

There was a discussion the other day about how Python devs apparently don't care enough for backwards compatibility. I pointed out that I've often gotten Python 2 code running on Python 3 by just changing print to print().

But then a few hours later, I tried running a very small project I wrote last year and it turned out that a bunch of my dependencies had changed their APIs. I've had similar (and much worse) experiences trying to get older code with dependencies running.

My meaning with this comment is, that if the average developer's reality is that backwards compatibility isn't really a thing anyway, then we are already paying for that downside so we might as well get some upside there, is my reasoning.

Re: State of Python 3.13 performance: Free-threading

#26

Post by eigenspace »

dagmx wrote: post #23
eigenspace wrote: post #21

Earlier quoted context omitted.

Fair enough. You may be totally right here, as I mentioned I don't use Python much at all since like 2017 and haven't paid it much attention in a while. I retract my comment. Regarding breakage in 3.x, all I know is that I recall several times where I did a linux system update (rolling release), and that updated my Python to a newly released version which broke various things in my system. I'm pretty sure one of thes…

To your last point: it’s neither the language nor the packages but rather it’s the ABI. Python isn’t fully ABI stable (though it’s improved greatly) so you can’t just intermix compiled dependencies between different versions of Python. This is true for many packages in your distro as well.

Ah I see. Yeah, I guess I just think that for a language that's so dependent on FFI, instability in the ABI is defacto instability in the language as far as I'm concerned. But I understand that not everyone feels the same.

Almost every major noteworthy Python package uses the ABI, so instability there is going to constantly be felt ecosystem wide.

Re: State of Python 3.13 performance: Free-threading

#27

Post by Spivak »

jerf wrote: post #16
sneed_chucker wrote: post #11

Earlier quoted context omitted.

If JavaScript (V8) and PyPy can be fast, then CPython can be fast too. It's just that the CPython developers and much of the Python community sat on their hands for 15 years and said stuff like "performance isn't a primary goal" and "speed doesn't really matter since most workloads are IO-bound anyway".

In this context, V8 and PyPy aren't fast. Or at least, not generally; they may actually do well on this task because pure number tasks are the only things they can sometimes, as long as you don't mess them up, get to compiled language-like performance. But they don't in general to compiled language performance, despite common belief to the contrary.

Let's make this more concrete because assigning speed to languages is a fools errand. Python is doing a lot more per line of code than compiled languages to enable its very flexible semantics. In cases where this flexibility is desired you won't see much more performance in a compiled language as you'll have just implemented Python-like semantics on top of your compiled language— GObject is a good example of this. More famously this is Greenspun's tenth rule.

> Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.

But where this flexibility isn't required, which is a lot of performance sensitive number crunching code the cost of the flexibility bites you. You can't "turn it off" when you want control down to the instruction for a truly massive performance win. Which is why I think the model Python has of highly expressive and flexible language backed by high-performance compiled libraries is so successful.

Python will never be number crunching or parsing with the best of them because it would require essentially a whole new language to express the low-level constraints but for high-level code that relies on Python's semantics you can get performance wins that can't be accomplished just by switching to a compiled language. We've just taken the "embedded scripting language" and made it the primary interface.

Re: State of Python 3.13 performance: Free-threading

#28

Post by almostgotcaught »

andai wrote: post #24
eigenspace wrote: post #3

I don't really have a dog in this race as I don't use Python much, but this sort of thing always seemed to be of questionable utility to me. Python is never really going to be 'fast' no matter what is done to it because its semantics make most important optimizations impossible, so high performance "python" is actually going to always rely on restricted subsets of the language that don't actually match language's "re…

There was a discussion the other day about how Python devs apparently don't care enough for backwards compatibility. I pointed out that I've often gotten Python 2 code running on Python 3 by just changing print to print(). But then a few hours later, I tried running a very small project I wrote last year and it turned out that a bunch of my dependencies had changed their APIs. I've had similar (and much worse) experi…

> I pointed out that I've often gotten Python 2 code running on Python 3 by just changing print to print().

...

> I wrote last year and it turned out that a bunch of my dependencies had changed their APIs

these two things have absolutely nothing to do with each other - couldn't be a more apples to oranges comparison if you tried

Re: State of Python 3.13 performance: Free-threading

#29

Post by jillesvangurp »

eigenspace wrote: post #3

I don't really have a dog in this race as I don't use Python much, but this sort of thing always seemed to be of questionable utility to me. Python is never really going to be 'fast' no matter what is done to it because its semantics make most important optimizations impossible, so high performance "python" is actually going to always rely on restricted subsets of the language that don't actually match language's "re…

Why does python have to be slow? Improvements over the last few releases have made it quite a bit faster. So that kind of counters that a bit. Apparently it didn't need to be quite as slow all along. Other languages can be fast. So, why not python?

I think with the GIL some people are overreacting: most python code is single threaded because of the GIL. So removing it doesn't actually break anything. The GIL was just making the use of threads kind of pointless. Removing it and making a lot of code thread safe benefits people who do want to use threads.

It's very simple. Either you did not care about performance anyway and nothing really changes for you. You'd need to add threading to your project to see any changes. Unless you do that, there's no practical reason to disable the GIL for you. Or to re-enable that once disabled becomes the default. If your python project doesn't spawn threads now, it won't matter to you either way. Your code won't have deadlocking threads because it has only 1 thread and there was never anything to do for the GIL anyway. For code like that compatibility issues would be fairly minimal.

If it does use threads, against most popular advise of that being quite pointless in python (because of the GIL), you might see some benefits and you might have to deal with some threading issues.

I don't see why a lot of packages would break. At best some of them would be not thread safe and it's probably a good idea to mark the ones that are thread safe as such in some way. Some nice package management challenge there. And probably you'd want to know which packages you can safely use.

Re: State of Python 3.13 performance: Free-threading

#30

Post by eigenspace »

jillesvangurp wrote: post #29
eigenspace wrote: post #3

I don't really have a dog in this race as I don't use Python much, but this sort of thing always seemed to be of questionable utility to me. Python is never really going to be 'fast' no matter what is done to it because its semantics make most important optimizations impossible, so high performance "python" is actually going to always rely on restricted subsets of the language that don't actually match language's "re…

Why does python have to be slow? Improvements over the last few releases have made it quite a bit faster. So that kind of counters that a bit. Apparently it didn't need to be quite as slow all along. Other languages can be fast. So, why not python? I think with the GIL some people are overreacting: most python code is single threaded because of the GIL. So removing it doesn't actually break anything. The GIL was just…

> Why does python have to be slow?

Because the language's semantics promise that a bunch of insane stuff can happen at any time during the running of a program, including but not limited to the fields of classes changing at any time. Furthermore, they promise that their integers are aribtrary precision which are fundamentally slower to do operations with than fixed precision machine integers, etc.

The list of stuff like this goes on and on and on. You fundamentally just cannot compile most python programs to efficient machine code without making (sometimes subtle) changes to its semantics.

_________

> I don't see why a lot of packages would break. At best some of them would be not thread safe and it's probably a good idea to mark the ones that are thread safe as such in some way. Some nice package management challenge there. And probably you'd want to know which packages you can safely use.

They're not thread safe because it was semantically guaranteed to them that it was okay to write code that's not thread safe.

Post reply on HN