Live data from Hacker News

Performance of the Python 3.14 tail-call interpreter

blog.nelhage.com

21–30 of 180 posts

Re: Performance of the Python 3.14 tail-call interpreter

#21
I recently made some benchmarking from python 3.9 to 3.13 And up to 3.11 it only got better. Python 3.12 and 3.13 were about 10% slower than 3.11.

I thought my homemade benchmark wasn't great enough so I deployed it to a core service anyway and I saw same changes in our collected metrics. Does anyone else have the same problem?

Re: Performance of the Python 3.14 tail-call interpreter

#22
post #15

Earlier quoted context omitted.

> The C expression `a += 1` can be trusted to increment a numeric value, [...] Have you heard of undefined behaviour?

Show me a C compiler that miscompiles the following code and I'll concede the point: uint32_t add_1(uint32_t a) { a += 1; return a; }

You show one example where C doesn't have problems, but that's a much weaker claim than it sounds. "Here's one situation where this here gun won't blow your foot off!"

For what it's worth, C++ also passes your test here. You picked an example so simple that it's not very interesting.

Re: Performance of the Python 3.14 tail-call interpreter

#23
post #22

Earlier quoted context omitted.

Show me a C compiler that miscompiles the following code and I'll concede the point: uint32_t add_1(uint32_t a) { a += 1; return a; }

You show one example where C doesn't have problems, but that's a much weaker claim than it sounds. "Here's one situation where this here gun won't blow your foot off!" For what it's worth, C++ also passes your test here. You picked an example so simple that it's not very interesting.

Actually even here, C has some problems (and C++), too:

I don't think the standard says much about how to handle stack overflows?

Re: Performance of the Python 3.14 tail-call interpreter

#24

This is a very good example of how C is not "close to the machine" or "portable assembly", modern optimizers will do drastic changes to the logic as long as it has no observable effect. As stated in the post: "Thus, we end up in this odd world where clang-19 compiles the computed-goto interpreter “correctly” – in the sense that the resulting binary produces all the same value we expect – but at the same time it produ…

> This is a very good example of how C is not "close to the machine" or > "portable assembly", C is very much "portable assembly" from the perspective of other systems programming languages of the 80s-90s era. The C expression `a += 1` can be trusted to increment a numeric value, but the same expression in C++ might allocate memory or unwind the call stack or do who knows what. Similarly, `a = "a"` is a simple pointe…

When the code has hit the IR in clang or gcc, there is no 'a' (we know that with certainty, since SSA form doesn't mutate but assigns to fresh variables). We don't know if there will be an increment of 1, the additions could be coalesced (or elided if the result can be inferred another way). The number can even decrease, say if things have been handled in chunks of 16, and needs to be adjusted down in the last chunk. Or the code may be auto-vectorized and completely rewritten, so that none of the variables at the C level are reflected on the assembler level.

Re: Performance of the Python 3.14 tail-call interpreter

#25
post #20

Earlier quoted context omitted.

Show me a C compiler that miscompiles the following code and I'll concede the point: uint32_t add_1(uint32_t a) { a += 1; return a; }

I know that for 'int a' the statement 'a += 1' can give rather surprising results. And you made a universal statement that 'a += 1' can be trusted. Not just that it can sometimes be trusted. In C++ the code you gave above can also be trusted as far as I can tell. At least as much as the C version.

I'll expand my point to be clearer.

In C there is no operator overloading, so an expression like `a += 1` is easy to understand as incrementing a numeric value by 1, where that value's type is one of a small set of built-in types.

You'd need to look further up in the function (and maybe chase down some typedefs) to see what that type is, but the set of possible types generally boils down to "signed int, unsigned int, float, pointer". Each of those types has well-defined rules for what `+= 1` means.

That means if you see `int a = some_fn(); assert(a ---

Neither of those is true in C++. The combination of completely ad-hoc operator overloading, function overloading, and implicit type conversion via constructors means that it can be really difficult to map between the original source and the machine code.

You'll have a core dump where EIP is somewhere in the middle of a function like this:

  std::string some_fn() {
    some_ns::unsigned a = 1;
    helper_fn(a, "hello");
    a += 1;
    return true;
  }
and the disassembly is just dozens of function calls for no reason you can discern, and you're staring at the return type of `std::string` and the returned value of `true`, and in that moment you'll long for the happy days when undefined behavior on signed integer overflow was the worst you had to worry about.

Re: Performance of the Python 3.14 tail-call interpreter

#26
post #14

Benchmarking is just insanely hard to do well. There are so many things which can mislead you. I recently discovered a way to make an algorithm about 15% faster. At least, that's what all the benchmarks said. At some point I duplicated the faster function in my test harness, but did not call the faster version, just the original slower one... And it was still 15% faster. So code that never executed sped up the origin…

I vaguely remember about some benchmarking project that deliberately randomised these compiler decisions, so that they could give you more stable estimates of how well your code actually performed, and not just how well you won or lost the linker lottery.

As already mentioned this is likely Emery Berger’s project with the idea of intentionally slowing down different parts of the program, also to find out which parts are most sensitive to slowdowns (aka have the biggest effect on overall performance), with the assumption that these are also the parts that profit the most from optimisations.

Re: Performance of the Python 3.14 tail-call interpreter

#27

Benchmarking is just insanely hard to do well. There are so many things which can mislead you. I recently discovered a way to make an algorithm about 15% faster. At least, that's what all the benchmarks said. At some point I duplicated the faster function in my test harness, but did not call the faster version, just the original slower one... And it was still 15% faster. So code that never executed sped up the origin…

That linker lottery led to a 15% improvement? I'm surprised. Do you know in what cases you get such a big improvement from something like that? Is it rare? How did you end up reasoning about it?

Re: Performance of the Python 3.14 tail-call interpreter

#28
post #15

Earlier quoted context omitted.

> The C expression `a += 1` can be trusted to increment a numeric value, [...] Have you heard of undefined behaviour?

Show me a C compiler that miscompiles the following code and I'll concede the point: uint32_t add_1(uint32_t a) { a += 1; return a; }

C might be low level from the perspective of other systems languages, but that is like calling Apollo 11 simple from the perspective of modern spacecraft. C as written is not all that close to what actually gets executed.

For a small example, there are many compilers who would absolutely skip incrementing 'a' in the following code:

  uint32_t add_and_subtract_1(uint32_t a) {
    a += 1;
    a -= 1;
    return a;
  }
Even though that code contains `a += 1;` clear as day, the chances of any incrementing being done are quite small IMO. It gets even worse in bigger functions where out-of-order execution starts being a thing.

Re: Performance of the Python 3.14 tail-call interpreter

#29
post #20

Earlier quoted context omitted.

I know that for 'int a' the statement 'a += 1' can give rather surprising results. And you made a universal statement that 'a += 1' can be trusted. Not just that it can sometimes be trusted. In C++ the code you gave above can also be trusted as far as I can tell. At least as much as the C version.

I'll expand my point to be clearer. In C there is no operator overloading, so an expression like `a += 1` is easy to understand as incrementing a numeric value by 1, where that value's type is one of a small set of built-in types. You'd need to look further up in the function (and maybe chase down some typedefs) to see what that type is, but the set of possible types generally boils down to "signed int, unsigned int,…

I heartily agree that C++ is a lot more annoying here than C, yes.

I'm just saying that C is already plenty annoying enough by itself, thanks eg to undefined behaviour.

> That means if you see `int a = some_fn(); assert(a No, there's no guarantee of that. C compilers are allowed to do all kinds of interesting things. However you are often right enough in practice, especially if you run with -O0, ie turn off the optimiser.

See eg https://godbolt.org/z/YY69Ezxnv and tell me where the ADD instruction shows up in the compiler output.

Re: Performance of the Python 3.14 tail-call interpreter

#30
post #20

Earlier quoted context omitted.

Show me a C compiler that miscompiles the following code and I'll concede the point: uint32_t add_1(uint32_t a) { a += 1; return a; }

I know that for 'int a' the statement 'a += 1' can give rather surprising results. And you made a universal statement that 'a += 1' can be trusted. Not just that it can sometimes be trusted. In C++ the code you gave above can also be trusted as far as I can tell. At least as much as the C version.

a+=1 will not produce any surprising results, signed integer overflow is well defined on all platforms that matter.

And we all know about the looping behavior, it isn't surprising.

The only surprising part would be if the compiler decides to use inc vs add, not that it really matters to the result.

Post reply on HN