Python 3.11: “Zero cost” exception handling • Hacker News Board

Re: Python 3.11: “Zero cost” exception handling

#71

Post by fouric » Wed, Oct 06, 2021, 6:10 PM UTC

BiteCode_dev wrote: post #46

albertzeyer wrote: post #45

Earlier quoted context omitted.

> exceptions are a fundamental control-flow mechanism in Python Where do you have this from? In almost all Python code I have seen, exceptions are still for, as the name says, exceptions. So in the common cases, there would be no exceptions raised. I would assume that the exception handling code (under `except ...`) will be run only a tiny fraction of times compared to the other code, at least in most cases. I would…

The for loop in python is a try/except catching StopIteration in a trench coat. Also, EAFP, and the context manager protocol.

> The for loop in python is a try/except catching StopIteration in a trench coat.

That's...horrifying, both from a computational efficiency perspective (due to the overhead of exceptions) and from a "beauty" perspective (exceptions are supposed to be exceptional, darn it!). Why is it implemented this way? Why not just use a normal loop and check a return value from the iterator?

Re: Python 3.11: “Zero cost” exception handling

#72

Post by dragonwriter » Wed, Oct 06, 2021, 6:14 PM UTC

fouric wrote: post #71

BiteCode_dev wrote: post #46

Earlier quoted context omitted.

The for loop in python is a try/except catching StopIteration in a trench coat. Also, EAFP, and the context manager protocol.

> The for loop in python is a try/except catching StopIteration in a trench coat. That's...horrifying, both from a computational efficiency perspective (due to the overhead of exceptions) and from a "beauty" perspective (exceptions are supposed to be exceptional , darn it!). Why is it implemented this way? Why not just use a normal loop and check a return value from the iterator?

> Why not just use a normal loop and check a return value from the iterator?

Because an iterator needs to be able to return any python value, and if you did that, there would be at least one python value that an iterator could not actually return.

Unless you required the iterator to wrap real return values in a container (basically, an optional/maybe monad) but then you have unwrapping overhead on every real return value, which is worse.

Re: Python 3.11: “Zero cost” exception handling

#73

Post by masklinn » Wed, Oct 06, 2021, 6:15 PM UTC

alex_smart wrote: post #66

masklinn wrote: post #9

Earlier quoted context omitted.

> In a language where idiomatic control flow uses exceptions? That's crazy! Seems like the opposite: if exceptions are extremely rare then you want to optimise the case where they’re not raised, at the cost of the other one. If exceptions are common then it matters a lot less, you may even want to avoid 0ce depending on the impact on the raised case.

I don't think python has the mindset of "exceptions are extremely rare". That is probably what the OC meant by python being "a language where idiomatic control flow uses exceptions". As an example, every iterator in python signals its end by throwing a StopIteration exception. So, every "for x in iter" has the interpreter throwing and catching an exception.

> I don't think python has the mindset of "exceptions are extremely rare".

No it does not, that's my point: when exceptions are extremely rare, as in C++ or Java, having the "no exception" case be free at additional expense in the "exception" case is an excellent tradeoff, because the latter should happen extremely rarely so even if the costs are ridiculous you'll have more than made them by all the cases where you got the "no exception" run for free.

In Python where exceptions are common however, making one case free at the expense of the other is less interesting a proposition, and is more of a balancing act: you don't want to overly penalise exceptions-heavy code as that is considered normal and idiomatic.

Re: Python 3.11: “Zero cost” exception handling

#74

Post by omegalulw » Wed, Oct 06, 2021, 6:17 PM UTC

bregma wrote: post #22

ziml77 wrote: post #2

They weren't zero cost before? In a language where idiomatic control flow uses exceptions? That's crazy! I've felt weird using exceptions like that but I always assumed that CPython was optimized to minimize overhead of exceptions and exception handlers.

Exceptions always have a cost. What happened with many C++ runtimes is that they moved the execution cost of exceptions almost entirely into the exception raising mechanism so that the path of execution that does not raise exceptions has no overhead due to exceptions. This was not the case with the Python runtime. The "zero cost" implementation for C++ exceptions in the case of ELF binaries means storing a bunch of s…

Thanks for the informative post.

> then the stack is unwound by interpreting the function table contents to step through the various CPU states in reverse order

Why do you need the exception function table contents for this. Presumably, you only want to destroy stack objects so isn't the call stuck necessary and sufficient to unroll all the way back to the function with the catch handler?

Also what are your thoughts on returning errors via a status vs exceptions. I love the former - they force you to somehow handle each error (pass it up or deal with it) vs exceptions where there is no accountability for a method that causally caused the exception.

Re: Python 3.11: “Zero cost” exception handling

#75

Post by wahern » Wed, Oct 06, 2021, 6:20 PM UTC

sonthonax wrote: post #67

bregma wrote: post #22

Earlier quoted context omitted.

Exceptions always have a cost. What happened with many C++ runtimes is that they moved the execution cost of exceptions almost entirely into the exception raising mechanism so that the path of execution that does not raise exceptions has no overhead due to exceptions. This was not the case with the Python runtime. The "zero cost" implementation for C++ exceptions in the case of ELF binaries means storing a bunch of s…

What’s a DSO?

Dynamic shared object, or what's usually understood generically (in non-language specific terms) as a [binary] module or in GUI apps a plugin--something loaded and linked dynamically by the program, not by the linker at compile time or at program startup. In the Unix/C ecosystem "DSO" is a common term to describe binary modules, without reference to specific binary formats like ELF or mach-O.

Especially in the land of Unix where dynamic linking has become very sophisticated and automated, many of these terms are losing their distinctive meanings as the various ways to link and load converge on the same underlying mechanisms which you can mix-and-match independently. But normally the difference between a DSO and a shared library is that shared library symbols (functions, global variables) are by default visible globally (at least on ELF; mach-O story is more complex), making them implicitly visible to subsequently loaded code, whether a shared library or DSO. On the other hand, you usually want all DSO symbols to remain private to the DSO by default, requiring the application at runtime to explicitly request a pointer to specific symbols as different DSOs might have symbols of the same name (especially true for plugins), leaving the application logic to decide which to use and when. Also, especially pertinent to Lua, Perl, Python, Ruby, and similar languages, a DSO usually implicitly relies on the main program to have loaded and exported certain core APIs, rather than the DSO explicitly linking to a shared library to provide them. (This still often requires special compile-time flags when building the DSO, and possibly also special flags to the main binary.) That ensures the VM/interpreter/engine and all DSOs are always calling into the same core runtime implementation, and ensures the main binary controls which that is. DSOs do often dynamically link to other shared libraries, but because of default symbol visibility semantics this can cause problems, such as two DSOs or a DSO and the main program compiled against different releases of OpenSSL with incompatible ABIs. (If you can manage to load two different version of OpenSSL, you have the reverse issue of accidentally passing objects between the two implementations.) Mitigations and resolutions for that problem differ between ELF-based and mach-O-based systems. (AIX is an outlier, which like Windows uses a PE-derived binary format that is significantly different than ELF or mach-O. Many of the semantic distinctions and jargon that developed around ELF don't make much sense in the context of PE.)

Re: Python 3.11: “Zero cost” exception handling

#76

Post by kzrdude » Wed, Oct 06, 2021, 6:26 PM UTC

ok123456 wrote: post #70

cnband wrote: post #21

The tenacity of people getting excited over micro optimizations in Python for more than two decades is remarkable. Nothing has happened despite monumental speed programs that were broadly advertised and marketed to corporations. Meanwhile, SBCL has an industrial strength compiler that predates Python and its trademark (the SBCL compiler was called "Python" before the trademark, thereby invalidating it). Python (the l…

It's when you put all these small optimizations together that it leads to something remarkable. It's analogous to video codecs: there are a bunch of individual optimizations that alone don't look that impressive, only saving ~1% here or there. But once they all are working together, you see savings of 10-50%.

I'm looking forward o seeing numbers per python release, for what the optimizations result in taken together. Mark Shannon is going at it with optimizations now.

Re: Python 3.11: “Zero cost” exception handling

#77

Post by Scramblejams » Wed, Oct 06, 2021, 6:30 PM UTC

fouric wrote: post #71

BiteCode_dev wrote: post #46

Earlier quoted context omitted.

The for loop in python is a try/except catching StopIteration in a trench coat. Also, EAFP, and the context manager protocol.

> The for loop in python is a try/except catching StopIteration in a trench coat. That's...horrifying, both from a computational efficiency perspective (due to the overhead of exceptions) and from a "beauty" perspective (exceptions are supposed to be exceptional , darn it!). Why is it implemented this way? Why not just use a normal loop and check a return value from the iterator?

exceptions are supposed to be exceptional, darn it

This feeling is at odds with what is generally considered idiomatic Python, for better or worse.

Upthread BiteCode mentioned EAFP. See, for example: https://devblogs.microsoft.com/python/idiomatic-python-eafp-...

(It's certainly not unanimous. Guido himself is on the record as saying EAFP isn't better than LBYL, but in most of the Python I've cruised through these last umpteen years [that wasn't some other language written in Python], I've certainly seen an overall preference for EAFP. Naturally YMMV.)

Re: Python 3.11: “Zero cost” exception handling

#78

Post by bregma » Wed, Oct 06, 2021, 6:36 PM UTC

omegalulw wrote: post #74

bregma wrote: post #22

Earlier quoted context omitted.

Exceptions always have a cost. What happened with many C++ runtimes is that they moved the execution cost of exceptions almost entirely into the exception raising mechanism so that the path of execution that does not raise exceptions has no overhead due to exceptions. This was not the case with the Python runtime. The "zero cost" implementation for C++ exceptions in the case of ELF binaries means storing a bunch of s…

Thanks for the informative post. > then the stack is unwound by interpreting the function table contents to step through the various CPU states in reverse order Why do you need the exception function table contents for this. Presumably, you only want to destroy stack objects so isn't the call stuck necessary and sufficient to unroll all the way back to the function with the catch handler? Also what are your thoughts…

You need to restore some execution context before invoking the destructors. The value of the implicit "this" parameter, for example, has to be set correctly before the call to the destructor. Depending on the CPU and calling conventions, other registers may need their values contextually restored.

Then there's the stack crawling itself. Again, depending on the CPU and calling conventions, just finding where the return value is stored for a function can require some gymnastics (aarch64 I'm looking at you here).

As to returning and handling errors via status returns instead of exceptions: you're factoring the cost of error handling into the good path. It's the opposite of zero-cost. I've never understood the argument that every piece of code needs to be able to handle every error that every subroutine it calls could ever encounter. No one even does that, and the end result is that "handle error by return codes" is usually faster than using exceptions, because the error handling ends up being not done at all. At least, that's my experience over 40 years of maintaining other people's code.

As to the performance of languages that force error status returns to be dealt with vs. exception handling? Profile. Get some numbers. Examine the bias of the person generating the numbers.

Re: Python 3.11: “Zero cost” exception handling

#79

Post by bregma » Wed, Oct 06, 2021, 6:40 PM UTC

Diggsey wrote: post #53

bregma wrote: post #22

Earlier quoted context omitted.

Exceptions always have a cost. What happened with many C++ runtimes is that they moved the execution cost of exceptions almost entirely into the exception raising mechanism so that the path of execution that does not raise exceptions has no overhead due to exceptions. This was not the case with the Python runtime. The "zero cost" implementation for C++ exceptions in the case of ELF binaries means storing a bunch of s…

Also, it's not really the case that exceptions (when not thrown) are zero-cost in C++, and not because of the instruction cache or increase to static data size. The cost is that exception-throwing functions inhibit many of the optimizations performed by compilers, so they generate worse code, even though no "extra" code is actually executed.

Do they? I spend a lot of time dealing with the optimizers in GCC and haven't noticed that happening. In GCC, a throw is implemented as a single function call to the C++ runtime function __cxa_throw(). If a simple single function call will pessimize performance we're screwed as a species.

Re: Python 3.11: “Zero cost” exception handling

#80

Post by albertzeyer » Wed, Oct 06, 2021, 6:55 PM UTC

BiteCode_dev wrote: post #46

albertzeyer wrote: post #45

Earlier quoted context omitted.

> exceptions are a fundamental control-flow mechanism in Python Where do you have this from? In almost all Python code I have seen, exceptions are still for, as the name says, exceptions. So in the common cases, there would be no exceptions raised. I would assume that the exception handling code (under `except ...`) will be run only a tiny fraction of times compared to the other code, at least in most cases. I would…

The for loop in python is a try/except catching StopIteration in a trench coat. Also, EAFP, and the context manager protocol.

But that is what I meant, this StopIteration would not be affected by the zero cost optimization. The zero cost optimization is about compiled Python bytecode and the VM interpreter. The StopIteration is handled internally by CPython (not via Python bytecode).

For example, consider this function:

    def foo():
        for i in range(3):
            print(i)

There is no try-except in this code, and neither in its bytecode. So this is not affected by the zero cost optimization from the linked issue.

The bytecode is this:

  2           0 LOAD_GLOBAL              0 (range)
              2 LOAD_CONST               1 (3)
              4 CALL_FUNCTION            1
              6 GET_ITER
        >>    8 FOR_ITER                12 (to 22)
             10 STORE_FAST               0 (i)

  3          12 LOAD_GLOBAL              1 (print)
             14 LOAD_FAST                0 (i)
             16 CALL_FUNCTION            1
             18 POP_TOP
             20 JUMP_ABSOLUTE            8
        >>   22 LOAD_CONST               0 (None)
             24 RETURN_VALUE

The iteration logic is inside the `FOR_ITER` op. And that is purely handled inside CPython.

Also, this is again what I meant: In user code, you rarely would use custom exceptions for control flow. Please show me any example where you have seen otherwise.

So, again, what I wrote: I'm very confident that for most user code, the code path under `except ...:` is only rarely executed. Please show me any big Python project where this would not be the case. I doubt that there is any.

Also, btw, on EAFP, Guido van Rossum disagrees: https://mail.python.org/pipermail/python-dev/2014-March/1331...