Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Performance of the Python 3.14 tail-call interpreter

blog.nelhage.com

Post reply on HN
Print view View original

121–130 of 180 posts

Re: Performance of the Python 3.14 tail-call interpreter

#121

Post by haberman »

kenjin4096 wrote: post #51

Hello. I'm the author of the PR that landed the tail-calling interpreter in CPython. First, I want to say thank you to Nelson for spending almost a month to get to the root of this issue. Secondly, I want to say I'm extremely embarrassed and sorry that I made such a huge oversight. I, and probably the rest of the CPython team did not expect the compiler we were using for the baseline to have that bug. I posted an apo…

I think it's important to note that a primary motivation of the tail call interpreter design is to be less vulnerable to the whims of the optimizer. From my original blog article about this technique (https://blog.reverberate.org/2021/04/21/musttail-efficient-i...):

> Theoretically, this control flow graph paired with a profile should give the compiler all of the information it needs to generate the most optimal code [for a traditional switch()-based interpreter]. In practice, when a function is this big and connected, we often find ourselves fighting the compiler. It spills an important variable when we want it to keep it in a register. It hoists stack frame manipulation that we want to shrink wrap around a fallback function invocation. It merges identical code paths that we wanted to keep separate for branch prediction reasons. The experience can end up feeling like trying to play the piano while wearing mittens.

That second-to-last sentence is exactly what has happened here. The "buggy" compiler merged identical code paths, leading to worse performance.

The "fixed" compiler no longer does this, but the fix is basically just tweaking a heuristic inside the compiler. There's no actual guarantee that this compiler (or another compiler) will continue to have the heuristic tweaked in the way that benefits us the most.

The tail call interpreter, on the other hand, lets us express the desired machine code pattern in the interpreter itself. Between "musttail", "noinline", and "preserve_none" attributes, we can basically constrain the problem such that we are much less at the mercy of optimizer heuristics.

For this reason, I think the benefit of the tail call interpreter is more than just a 3-5% performance improvement. It's a reliable performance improvement that may be even greater than 3-5% on some compilers.

Re: Performance of the Python 3.14 tail-call interpreter

#122

Post by sundbry »

kenjin4096 wrote: post #51

Hello. I'm the author of the PR that landed the tail-calling interpreter in CPython. First, I want to say thank you to Nelson for spending almost a month to get to the root of this issue. Secondly, I want to say I'm extremely embarrassed and sorry that I made such a huge oversight. I, and probably the rest of the CPython team did not expect the compiler we were using for the baseline to have that bug. I posted an apo…

You don't have to apologize, you did great work either way.

Re: Performance of the Python 3.14 tail-call interpreter

#123

Post by ehsankia »

jraph wrote: post #71
kenjin4096 wrote: post #51

Hello. I'm the author of the PR that landed the tail-calling interpreter in CPython. First, I want to say thank you to Nelson for spending almost a month to get to the root of this issue. Secondly, I want to say I'm extremely embarrassed and sorry that I made such a huge oversight. I, and probably the rest of the CPython team did not expect the compiler we were using for the baseline to have that bug. I posted an apo…

Reading that you are extremely embarrassed and sorry that you made such a huge oversight, I was imagining you had broken something / worsened CPython's performance. But it's nothing like this. You announced a 10-15% perf improvement but that improvement is more like 1-5% on a non buggy compiler. It's not even like that 10-15% figure is wrong , it's just that it's correct only under very specific conditions, unknowing…

In some way, by indirectly helping fix this bug, they led to a ~10% performance increase for everyone who was using that faulty compiler! That's even better than an optional flag that many people won't know about or use.

Re: Performance of the Python 3.14 tail-call interpreter

#124

Post by ram_rar »

Kudo to author and Cpython team to reckon this. Nevertheless, its still a very important improvement in +Ve direction. Thats all matters. This a great story as to why benchmarking are soo hard to get right. I always tell my team to benchmark your real world use case as closely as possible and not rely on external results.

Re: Performance of the Python 3.14 tail-call interpreter

#125

Post by DannyBee »

theLiminator wrote: post #118
DannyBee wrote: post #76

Earlier quoted context omitted.

FWIW - the fix was merged since you wrote that blog post ;) Beyond that - 3-5% is a lot for something as old as the python interpreter if it holds up. I would still be highly proud of that. After 30 years, i've learned (like i expect you have) to be suspicious of any significant (IE >1%) performance improvement in a system that has existed a long time. They happen for sure, but are less common. Often, people are shif…

> After 30 years, i've learned (like i expect you have) to be suspicious of any significant (IE >1%) performance improvement in a system that has existed a long time. Laughs in corporate code

Sure, let me amend it to "i'm suspicious of any significant performance improvement in as system where performance actually matters, and has existed in a state where performance matters for a long time".

Re: Performance of the Python 3.14 tail-call interpreter

#126

Post by ehsankia »

jeeybee wrote: post #3

Kudos to the author for diving in and uncovering the real story here. The Python 3.14 tail-call interpreter is still a nice improvement (any few-percent gain in a language runtime is hard-won), just not a magic 15% free lunch. More importantly, this incident gave us valuable lessons about benchmarking rigor and the importance of testing across environments. It even helped surface a compiler bug that can now be fixed…

I guess the bigger question for me is, how was a 10% drop in Python performance not detected when that faulty compiler feature was pushed? Do we not benchmark the compilers themselves? Do the existing benchmarks on the compiler or python side not use that specific compiler?

Re: Performance of the Python 3.14 tail-call interpreter

#127

Post by immibis »

kenjin4096 wrote: post #51

Hello. I'm the author of the PR that landed the tail-calling interpreter in CPython. First, I want to say thank you to Nelson for spending almost a month to get to the root of this issue. Secondly, I want to say I'm extremely embarrassed and sorry that I made such a huge oversight. I, and probably the rest of the CPython team did not expect the compiler we were using for the baseline to have that bug. I posted an apo…

You don't need a long heartfelt apology - a simple "god dammit" is enough for this. You made no mistake - you only got unlucky.

It's possible to improve your luck by applying more care, but it's also possible to apply so much care that you do not try things that would have been useful, so I'd rather you keep erring on the side that you did!

Re: Performance of the Python 3.14 tail-call interpreter

#128

Post by colechristensen »

theLiminator wrote: post #118
DannyBee wrote: post #76

Earlier quoted context omitted.

FWIW - the fix was merged since you wrote that blog post ;) Beyond that - 3-5% is a lot for something as old as the python interpreter if it holds up. I would still be highly proud of that. After 30 years, i've learned (like i expect you have) to be suspicious of any significant (IE >1%) performance improvement in a system that has existed a long time. They happen for sure, but are less common. Often, people are shif…

> After 30 years, i've learned (like i expect you have) to be suspicious of any significant (IE >1%) performance improvement in a system that has existed a long time. Laughs in corporate code

I'll believe you if you say 0.5% improvement, I'll believe you if you say 10,000% improvement, but 10%? That's fishy.

Re: Performance of the Python 3.14 tail-call interpreter

#129

Post by gpderetta »

hun3 wrote: post #40
jmillikin wrote: post #34

Earlier quoted context omitted.

From a high-level academic view, yes, the compiler is allowed to perform any legal transformation. But in practice C compilers are pretty conservative about what they emit, especially when code is compiled without -march= . You don't have to take my word for it. Go find a moderately complex open-source library written in C, compile it, then open up the result in Hexrays/Ghidra/radare2/whatever. Compare the compiled f…

-O3 does autovectorization: turning your loops into a bunch of SIMD instructions, sometimes even drastically changing performance profile. If autovectorization is "not that much magic" then idk what else it is.

Any optimization you are familiar with is trivial and expected. Everything else is broken compilers optimizing UB to win benchmarks.

Re: Performance of the Python 3.14 tail-call interpreter

#130

Post by igouy »

eru wrote: post #14
MattPalmer1086 wrote: post #9

Benchmarking is just insanely hard to do well. There are so many things which can mislead you. I recently discovered a way to make an algorithm about 15% faster. At least, that's what all the benchmarks said. At some point I duplicated the faster function in my test harness, but did not call the faster version, just the original slower one... And it was still 15% faster. So code that never executed sped up the origin…

I vaguely remember about some benchmarking project that deliberately randomised these compiler decisions, so that they could give you more stable estimates of how well your code actually performed, and not just how well you won or lost the linker lottery.

"Producing wrong data without doing anything obviously wrong!"

https://doi.org/10.1145/1508244.1508275

Post reply on HN