61–70 of 180 posts
I recently made some benchmarking from python 3.9 to 3.13 And up to 3.11 it only got better. Python 3.12 and 3.13 were about 10% slower than 3.11. I thought my homemade benchmark wasn't great enough so I deployed it to a core service anyway and I saw same changes in our collected metrics. Does anyone else have the same problem?
Hello. I'm the author of the PR that landed the tail-calling interpreter in CPython. First, I want to say thank you to Nelson for spending almost a month to get to the root of this issue. Secondly, I want to say I'm extremely embarrassed and sorry that I made such a huge oversight. I, and probably the rest of the CPython team did not expect the compiler we were using for the baseline to have that bug. I posted an apo…
Why didn't this regression in baseline performance show up (or did it?) on the faster-cpython benchmarks page [0]? Could the benchmarks be improved to prevent similar issues in the future? [0] https://github.com/faster-cpython/benchmarking-public
Clang 19 was released last year. We only benched it a few months ago. We did notice there was a significant slowdown on macOS, but that was against Xcode Clang, which is a different compiler. I thought it might've been an Xcode thing, which in the past has bit CPython before (such as Xcode LTO working/not working versus normal Clang) so I didn't investigate deeper (facepalming now in retrospect) and chalked it up to a compiler difference.
TLDR: We didn't run benchmarks of clang 19 versus 18. We only ran benchmarks of clang 19 versus gcc, Xcode clang, and MSVC. All of which are not apples-to-apples to Clang 19, so I naiively thought it was a compiler difference.
EDIT: As to how we could improve this process, I'm not too sure, but I know I'll be more discerning when there's a >4% perf hit now when upgrading compilers.
Hello. I'm the author of the PR that landed the tail-calling interpreter in CPython. First, I want to say thank you to Nelson for spending almost a month to get to the root of this issue. Secondly, I want to say I'm extremely embarrassed and sorry that I made such a huge oversight. I, and probably the rest of the CPython team did not expect the compiler we were using for the baseline to have that bug. I posted an apo…
Hello. I'm the author of the PR that landed the tail-calling interpreter in CPython. First, I want to say thank you to Nelson for spending almost a month to get to the root of this issue. Secondly, I want to say I'm extremely embarrassed and sorry that I made such a huge oversight. I, and probably the rest of the CPython team did not expect the compiler we were using for the baseline to have that bug. I posted an apo…
I just wanted to say: respect for being able to say "sorry, I made a mistake". I hate the fake it till you make it mentality that seems to be the norm now.
Why do I even bring this message - I want to say that let's not let what we see in the news influence our perception of the real people of the world. Just because fraud and crimes get elevated in the news, does not mean that the common man is a criminal or a fraud. :)
Hello. I'm the author of the PR that landed the tail-calling interpreter in CPython. First, I want to say thank you to Nelson for spending almost a month to get to the root of this issue. Secondly, I want to say I'm extremely embarrassed and sorry that I made such a huge oversight. I, and probably the rest of the CPython team did not expect the compiler we were using for the baseline to have that bug. I posted an apo…
One question arises: does the added code [1] bring any improvement, or does it merely complicate the source? Should it not be removed? [1] https://github.com/python/cpython/pull/128718
The blog post mentions it brings a 1-5% perf improvement. Which is still significant for CPython. It does not complicate the source because we use a DSL to generate CPython's interpreters. So the only complexity is in autogenerated code, which is usually meant for machine consumption anyways.
The other benefit (for us maintainers I guess), is that it compiles way faster and is more debuggable (perf and other tools work better) when each bytecode is a smaller function. So I'm inclined to keep it for perf and productivity reasons.
The free-threading had massive regressions (50% slower), this one has almost no speedups.
People are scared of corporate repressions or repressions from the Steering Council, so they shut up like in the Soviet Union.
Hello. I'm the author of the PR that landed the tail-calling interpreter in CPython. First, I want to say thank you to Nelson for spending almost a month to get to the root of this issue. Secondly, I want to say I'm extremely embarrassed and sorry that I made such a huge oversight. I, and probably the rest of the CPython team did not expect the compiler we were using for the baseline to have that bug. I posted an apo…
Why didn't this regression in baseline performance show up (or did it?) on the faster-cpython benchmarks page [0]? Could the benchmarks be improved to prevent similar issues in the future? [0] https://github.com/faster-cpython/benchmarking-public
As alluded to in https://news.ycombinator.com/item?id=43319010, I see these tests were collected against just 2 Intel and 2 ARM CPUs. So, if you are looking for feedback to improve, you should probably also include (at least) a AMD Zen4 or Zen5 in there. CPU & compiler people have been both trying to "help perf while not trusting the other camp" for as long as I can remember and I doubt that problem will ever go away.
A couple more CPUs will help but not solve generalizability of results. E.g., if somebody tests against some ancient 2008 Nehalem hardware, they might get very different answers. Similarly, answers today might not reflect 2035 very well.
The reality of our world of complex hardware deployment (getting worse with GPUs) is that "portable performance" is almost a contradiction in terms. We all just do the best we can at some semblance of the combination. The result is some kind of weighted average of "not #ifdef'd too heavily source" and "a bit" faster "informally averaged over our user population & their informally averaged workloads" and this applies at many levels of the whole computing stack.
EDIT: And, of course, a compiled impl like Cython or Nim is another way to go if you care about performance, but I do understand the pull & network effects of the Python ecosystem. So, that may not always be practical.
Benchmarking is just insanely hard to do well. There are so many things which can mislead you. I recently discovered a way to make an algorithm about 15% faster. At least, that's what all the benchmarks said. At some point I duplicated the faster function in my test harness, but did not call the faster version, just the original slower one... And it was still 15% faster. So code that never executed sped up the origin…
That linker lottery led to a 15% improvement? I'm surprised. Do you know in what cases you get such a big improvement from something like that? Is it rare? How did you end up reasoning about it?
I got the benchmarking tool itself to give reasonably repeatable measurements.
The tool had high precision, but the accuracy of "which algorithm is better" was not reliable just due to these code layout issues.
I basically gave up and shelved the benchmarking project at that point, because it wasn't actually useful to determine which algorithm was better.
Earlier quoted context omitted.
Counterexample: https://godbolt.org/z/z3jqjPT6o
I'm not sure that's a counter-example -- what assembly do you think should be emitted for floating-point math on an AVR microcontroller?
More examples of non-trivial mapping from C code to generated code: https://godbolt.org/z/jab6vh6dM