Hah, I found something similar in gcc a few years back. Affecting postgres' expression evaluation interpreter:
Performance of the Python 3.14 tail-call interpreter
111–120 of 180 posts
Re: Performance of the Python 3.14 tail-call interpreter
#112Earlier quoted context omitted.
That's a fair question. The blog post mentions it brings a 1-5% perf improvement. Which is still significant for CPython. It does not complicate the source because we use a DSL to generate CPython's interpreters. So the only complexity is in autogenerated code, which is usually meant for machine consumption anyways. The other benefit (for us maintainers I guess), is that it compiles way faster and is more debuggable…
There was a plan for a 5x speedup overall looking for funding back in 2022. Then a team with Guido and others involved (and MS backing?) got on the same bandwagon and made some announcements for speeding up CPython a lot. Several releases in, have we seen even a 2x speedup? Or more like 0.2x at best? Not trying to dismiss the interpreter changes - more want to know if those speedup plans were even remotely realistic,…
The faster cpython project was for 5x over the 3.10 baseline. CPython 3.13 is currently running at something like 1.6x speed compared to 3.10. With the JIT enabled it goes up by a few more single digit percentage point. With the changes in 3.14 it'll be something like 1.8x speed-up.
So it's slowly getting there, I think the faster cpython project was mostly around the idea that the JIT can get a lot faster as it starts to optimise more and more and that only just got shipped in 3.13, so there's a lot of headroom. We know that PyPy (an existing JIT implementation) is close to 5x faster than CPython a lot of the time already.
There's also now the experimental free-threading build which speeds up multithreaded Python applications (Not by a lot right now though unfortunately).
Re: Performance of the Python 3.14 tail-call interpreter
#113Earlier quoted context omitted.
One question arises: does the added code [1] bring any improvement, or does it merely complicate the source? Should it not be removed? [1] https://github.com/python/cpython/pull/128718
That's a fair question. The blog post mentions it brings a 1-5% perf improvement. Which is still significant for CPython. It does not complicate the source because we use a DSL to generate CPython's interpreters. So the only complexity is in autogenerated code, which is usually meant for machine consumption anyways. The other benefit (for us maintainers I guess), is that it compiles way faster and is more debuggable…
Being more robust to fragile compiler optimizations is also a nontrivial benefit. An interpreter loop is an extremely specialized piece of code whose control flow is too important to be left to compiler heuristics.
If the desired call structure can be achieved in a portable way, that's a win IMO.
Re: Performance of the Python 3.14 tail-call interpreter
#114To clarify: The situation is still not completely understood? It's not just only the computed gotos, but there is some other regression in Clang19? Basically, the difference between clang19.nocg and clang19 is not really clear? Btw, what about some clang18.tc comparison, i.e. Clang18 with the new tail-call interpreter? I wonder how that compares to clang19.tc.
> Btw, what about some clang18.tc comparison
(post author here) Oh, this is something I could have called out explicitly: The tail-calling interpreter relies on a feature (the `preserve_none` calling convention) that only landed in clang-19. That means you can only test it on that version. That coincidence (that 19 added both this feature, and the regression) is part of why this was so easy to miss at first, and why I had to "triangulate" with so many different benchmarks to be confident I understood what was going on.
Re: Performance of the Python 3.14 tail-call interpreter
#115So, the compiler is tinkering with the way the loop is organised so the whole tail-call interpreter thing is not as effective as announced... Not surprised. 1. CPU arch (and arch version) matters a lot. The problem is 95% about laying out the instruction dispatching code for the branch predictor to work optimally. C was never meant to support this. 2. The C abstract machine is also not low-level enough to express the…
(author here)
> The problem is 95% about laying out the instruction dispatching code for the branch predictor to work optimally.
A fun fact I learned while writing this post is that that's no longer true! Modern branch predictors can pretty much accurately predict through a single indirect jump, if the run is long enough and the interpreted code itself has stable behavior!
Here's a paper that studied this (for both real hardware and a certain simulated branch predictor): https://inria.hal.science/hal-01100647/document
My experiments on this project anecdotally agree; they didn't make it into the post but I also explored a few of the interpreters through hardware CPU counters and `perf stat`, and branch misprediction never showed up as a dominant factor.
Re: Performance of the Python 3.14 tail-call interpreter
#116Benchmarking is just insanely hard to do well. There are so many things which can mislead you. I recently discovered a way to make an algorithm about 15% faster. At least, that's what all the benchmarks said. At some point I duplicated the faster function in my test harness, but did not call the faster version, just the original slower one... And it was still 15% faster. So code that never executed sped up the origin…
I vaguely remember about some benchmarking project that deliberately randomised these compiler decisions, so that they could give you more stable estimates of how well your code actually performed, and not just how well you won or lost the linker lottery.
LLD has a new option "--randomize-section-padding" for this purpose: https://github.com/llvm/llvm-project/pull/117653
Re: Performance of the Python 3.14 tail-call interpreter
#117So, the compiler is tinkering with the way the loop is organised so the whole tail-call interpreter thing is not as effective as announced... Not surprised. 1. CPU arch (and arch version) matters a lot. The problem is 95% about laying out the instruction dispatching code for the branch predictor to work optimally. C was never meant to support this. 2. The C abstract machine is also not low-level enough to express the…
(author here) > The problem is 95% about laying out the instruction dispatching code for the branch predictor to work optimally. A fun fact I learned while writing this post is that that's no longer true! Modern branch predictors can pretty much accurately predict through a single indirect jump, if the run is long enough and the interpreted code itself has stable behavior! Here's a paper that studied this (for both r…
Yes, this was already becoming true around the time I was writing the linked article. And I also read the paper. :-) I also remember I had access to a pre-Haswell era Intel CPUs vs something a bit more recent, and could see that the more complicated dispatcher no longer made as much sense.
Conclusion: the rise of popular interpreter-based languages lead to CPUs with smarter branch predictors.
What's interesting is that a token threaded interpreter dominated my benchmark (https://github.com/vkazanov/bytecode-interpreters-post/blob/...).
This trick is meant to simplify dispatching logic and also spread branches in the code a bit.
Re: Performance of the Python 3.14 tail-call interpreter
#118Hello. I'm the author of the PR that landed the tail-calling interpreter in CPython. First, I want to say thank you to Nelson for spending almost a month to get to the root of this issue. Secondly, I want to say I'm extremely embarrassed and sorry that I made such a huge oversight. I, and probably the rest of the CPython team did not expect the compiler we were using for the baseline to have that bug. I posted an apo…
FWIW - the fix was merged since you wrote that blog post ;) Beyond that - 3-5% is a lot for something as old as the python interpreter if it holds up. I would still be highly proud of that. After 30 years, i've learned (like i expect you have) to be suspicious of any significant (IE >1%) performance improvement in a system that has existed a long time. They happen for sure, but are less common. Often, people are shif…
> After 30 years, i've learned (like i expect you have) to be suspicious of any significant (IE >1%) performance improvement in a system that has existed a long time.
Laughs in corporate code
Re: Performance of the Python 3.14 tail-call interpreter
#119Earlier quoted context omitted.
I vaguely remember about some benchmarking project that deliberately randomised these compiler decisions, so that they could give you more stable estimates of how well your code actually performed, and not just how well you won or lost the linker lottery.
LLD has a new option "--randomize-section-padding" for this purpose: https://github.com/llvm/llvm-project/pull/117653
Interesting, thanks!
Re: Performance of the Python 3.14 tail-call interpreter
#120Earlier quoted context omitted.
FWIW - the fix was merged since you wrote that blog post ;) Beyond that - 3-5% is a lot for something as old as the python interpreter if it holds up. I would still be highly proud of that. After 30 years, i've learned (like i expect you have) to be suspicious of any significant (IE >1%) performance improvement in a system that has existed a long time. They happen for sure, but are less common. Often, people are shif…
> After 30 years, i've learned (like i expect you have) to be suspicious of any significant (IE >1%) performance improvement in a system that has existed a long time. Laughs in corporate code
That's a different case. Corporate code is never optimized for performance. Performance as a factor doesn't play any factor.