Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Performance of the Python 3.14 tail-call interpreter

blog.nelhage.com

Post reply on HN
Print view View original

141–150 of 180 posts

Re: Performance of the Python 3.14 tail-call interpreter

#141

Post by ncruces »

chowells wrote: post #134
ehsankia wrote: post #123

Earlier quoted context omitted.

In some way, by indirectly helping fix this bug, they led to a ~10% performance increase for everyone who was using that faulty compiler! That's even better than an optional flag that many people won't know about or use.

That performance regression only hit code that was using a very large number of paths with the same table of computed gotos at the end. That's likely to only be relatively complex interpreters that were affected. So it's not a broad performance improvement. But it is nice to have an example of the compiler's new heuristic failing to prove evidence it needs to be tunable.

Well, that includes at least everyone using Python built with that compiler.

Re: Performance of the Python 3.14 tail-call interpreter

#142

Post by ggu »

kenjin4096 wrote: post #51

Hello. I'm the author of the PR that landed the tail-calling interpreter in CPython. First, I want to say thank you to Nelson for spending almost a month to get to the root of this issue. Secondly, I want to say I'm extremely embarrassed and sorry that I made such a huge oversight. I, and probably the rest of the CPython team did not expect the compiler we were using for the baseline to have that bug. I posted an apo…

Hello, I was one of the people who contributed to this interpreter change. Thank you Nelson for the excellent write-up and for going to such lengths to get to the bottom of this. That said, I wanted to defend Ken a bit. :-)

Ken Jin is a volunteer who's been tirelessly working to make CPython faster over the past couple of years for not enough credit. IMHO, Ken did nothing wrong here, and there's really nothing to be embarrassed about. The fact that it took a month (more if you consider the folks helping to reproduce the original results) for anyone to notice anything wrong shows how complicated the situation was! Put yourself in Ken's shoes -- having updated to LLVM-19 to enable the preserve_none flag, would you then hypothesize that LLVM-19 may have introduced an unrelated performance regression that no one noticed for five months? Lots of people underestimate how hard benchmarking is IMO.

A 1-5% performance improvement is also pretty valuable, just not quite as spectacular as we thought originally :-)

Re: Performance of the Python 3.14 tail-call interpreter

#143

Post by dwattttt »

celeritascelery wrote: post #136
nelhage wrote: post #115

Earlier quoted context omitted.

(author here) > The problem is 95% about laying out the instruction dispatching code for the branch predictor to work optimally. A fun fact I learned while writing this post is that that's no longer true! Modern branch predictors can pretty much accurately predict through a single indirect jump, if the run is long enough and the interpreted code itself has stable behavior! Here's a paper that studied this (for both r…

How do you reconcile that with the observation that moving to a computed goto style provides better codegen in zig[1]? They make the claim that using their “labeled switch” (which is essentially computed goto) allows you to have multiple branches which improves branch predictor performance. They even get a 13% speedup in their parser from switch to this style. If modern CPU’s are good at predicting through a single b…

While it's unlikely as neat as this, the blog post we're all commenting on is a "I thought we had a 10-15% speedup, but it turned out to be an LLVM optimisation misbehaving". And Zig (for now) uses LLVM for optimised builds too

Re: Performance of the Python 3.14 tail-call interpreter

#144

Post by thunkingdeep »

bjourne wrote: post #52

This is exactly the kind of content I love to see on HN. But I wonder though how this optimization is related to tail-call optimization? How the interpreter jump table is implemented shouldn't affect how stack frames are created, should it?

Well if you’d bother to read it, you’d discover this is about tail calls in C, not in Python. It has nothing to do with tail recursion in Python. Guido has explicitly said that Python will never have it.

Re: Performance of the Python 3.14 tail-call interpreter

#146

Post by rtollert »

jraph wrote: post #71
kenjin4096 wrote: post #51

Hello. I'm the author of the PR that landed the tail-calling interpreter in CPython. First, I want to say thank you to Nelson for spending almost a month to get to the root of this issue. Secondly, I want to say I'm extremely embarrassed and sorry that I made such a huge oversight. I, and probably the rest of the CPython team did not expect the compiler we were using for the baseline to have that bug. I posted an apo…

Reading that you are extremely embarrassed and sorry that you made such a huge oversight, I was imagining you had broken something / worsened CPython's performance. But it's nothing like this. You announced a 10-15% perf improvement but that improvement is more like 1-5% on a non buggy compiler. It's not even like that 10-15% figure is wrong , it's just that it's correct only under very specific conditions, unknowing…

> IIUC, you did your homework: you made an improvement, you measured a 10-15% perf improvement, the PR was reviewed by other people, etc. It just so happens that this 10-15% figure is misleading because of an issue with the version of clang you happened to use to measure. Unless I'm missing something, it looks like a fair mistake anyone could have reasonably made. It even looks like it was hard to not fall into this trap. You could have been more suspicious seeing such a high number, but hindsight is 20/20.

Hah! Is this a Gettier problem [0]?

1. True: The PR improves Python performance 15-20%. 2. True: Ken believes that the PR improves Python performance 15-20%. 3. True: Ken is justified in believing that the PR improves Python performance 15-20%.

Of course, PR discussions don't generally revolve around whether or not the PR author "knows" that the PR does what they claim it does. Still: these sorts of epistemological brain teasers seem to come up in the performance measurement field distressingly often. I wholeheartedly agree that Ken deserves all the kudos he has received; still, I also wonder if some of the strategies used to resolve the Gettier problem might be useful for code reviewers to center themselves every once in a while. Murphy's Law and all that.

[0]: https://en.wikipedia.org/wiki/Gettier_problem

Re: Performance of the Python 3.14 tail-call interpreter

#147

Post by ok_dad »

kenjin4096 wrote: post #51

Hello. I'm the author of the PR that landed the tail-calling interpreter in CPython. First, I want to say thank you to Nelson for spending almost a month to get to the root of this issue. Secondly, I want to say I'm extremely embarrassed and sorry that I made such a huge oversight. I, and probably the rest of the CPython team did not expect the compiler we were using for the baseline to have that bug. I posted an apo…

Yea, don't feel too bad, you're just unlucky that your mistake was so public, most of us have these kinds of mistakes weekly in a more private setting. I think everyone who knows anything about how optimized Python is would be impressed that you managed even a few percent improvement in speed! That will also probably save many, many GWh of power, even those few percent!

You have a new account here and your blog is just one article so far so you might be a new-ish developer(?), but you are doing great, keep it up! If you are not a new developer, you are still doing great, keep it up!

Re: Performance of the Python 3.14 tail-call interpreter

#149

Post by twoodfin »

ehsankia wrote: post #126
jeeybee wrote: post #3

Kudos to the author for diving in and uncovering the real story here. The Python 3.14 tail-call interpreter is still a nice improvement (any few-percent gain in a language runtime is hard-won), just not a magic 15% free lunch. More importantly, this incident gave us valuable lessons about benchmarking rigor and the importance of testing across environments. It even helped surface a compiler bug that can now be fixed…

I guess the bigger question for me is, how was a 10% drop in Python performance not detected when that faulty compiler feature was pushed? Do we not benchmark the compilers themselves? Do the existing benchmarks on the compiler or python side not use that specific compiler?

The author makes this point, too, and I agree it’s the most surprising thing about the entire scenario.

LLVM introduced a major CPython performance regression, and nobody noticed for six months?

Re: Performance of the Python 3.14 tail-call interpreter

#150

Post by eru »

johnisgood wrote: post #94
vikramkr wrote: post #78

Earlier quoted context omitted.

I'm pretty sure that's replying directly to the comment about how c is close to assembly and that if you add that line of code somewhere you know there's a variable getting incremented. Doesn't really matter whether or not it's useful, the point is that the behavior isn't exactly what you wrote

https://godbolt.org/z/r39jK1ddv It increments, then decrements with -O0 though. I do not see the issue still, as the behavior is expected with -O0; increments then decrements.

There's nothing in the C standard that enforces the observed -O0 behaviour. Your compiler might change tomorrow.
Post reply on HN