Python 3.11: “Zero cost” exception handling • Hacker News Board

Re: Python 3.11: “Zero cost” exception handling

#31

Post by ghshephard » Wed, Oct 06, 2021, 2:30 PM UTC

zohch wrote: post #3

ziml77 wrote: post #2

They weren't zero cost before? In a language where idiomatic control flow uses exceptions? That's crazy! I've felt weird using exceptions like that but I always assumed that CPython was optimized to minimize overhead of exceptions and exception handlers.

> I've felt weird using exceptions like that How should they be used instead? Maybe I don't understand what you mean by "idiomatic control flow uses exceptions" - could you give an example. Maybe there is some use of exceptions that I'm not quite familiar with in Python.

This covers it very well: https://devblogs.microsoft.com/python/idiomatic-python-eafp-...

In particular, this is not idiomatic python:

    if "key" in dict_:
        value += dict_["key"]

But this is:

    try:
        value += dict_["key"]
    except KeyError:
        pass

I too hate using the exception handling in this way, and if you aren't careful, you end up papering over other unexpected exceptions in your code, so you have to be (A) very specific in the exception you catch, and (B) keep it in as small a portion of code as possible.

I just think it makes for clumsy code - which of the two look better:

    try:
        value = dict_["key"]
    except KeyError:
        pass
    else:
        do_something(value)

OR

    if "key" in dict_:
        do_something(dict["key"])

But it might just be me.

Re: Python 3.11: “Zero cost” exception handling

#32

Post by Someone » Wed, Oct 06, 2021, 2:31 PM UTC

bregma wrote: post #22

ziml77 wrote: post #2

They weren't zero cost before? In a language where idiomatic control flow uses exceptions? That's crazy! I've felt weird using exceptions like that but I always assumed that CPython was optimized to minimize overhead of exceptions and exception handlers.

Exceptions always have a cost. What happened with many C++ runtimes is that they moved the execution cost of exceptions almost entirely into the exception raising mechanism so that the path of execution that does not raise exceptions has no overhead due to exceptions. This was not the case with the Python runtime. The "zero cost" implementation for C++ exceptions in the case of ELF binaries means storing a bunch of s…

One of the stated benefits of this is

> Calls to Python functions would be faster as frame objects would be considerably smaller. Currently each frame carries 240 bytes of overhead for exception handling.

I guess that’s where this will pay even in a language/ecosystem where exceptions aren’t exceptional.

Re: Python 3.11: “Zero cost” exception handling

#33

Post by dec0dedab0de » Wed, Oct 06, 2021, 2:36 PM UTC

brianwawok wrote: post #11

ubercore wrote: post #4

Earlier quoted context omitted.

The first that came to mind is how `get()` is handled in Django's ORM. The idiomatic way to look for a single object is to use `get`, then catch a `DoesNotExist` exception: From https://docs.djangoproject.com/en/3.2/ref/models/querysets/#... from django.core.exceptions import ObjectDoesNotExist try: blog = Blog.objects.get(id=1) entry = Entry.objects.get(blog=blog, entry_number=1) except ObjectDoesNotExist: print("Ei…

Right, but the better way to actually write this is something like entry = Entry.objects.filter(blog__id=1, entry_number=1).first() if entry is None: # deal with does not exist Maybe it's my scala/Java background shining through, but we are big Django users and we ban the "catch exceptions as standard" workflow, because there is almost always a cleaner way...

I wouldn't make it an if statement unless it's going to be a part of the standard flow. I think the catchphrase is "leap before you look". Though your right that a single query is better.

Honestly, I normally just use get and let the exception fly. If it's a celery task I'll see the stack trace in flower, or right in the output if it's dev with debug on. Then I would go out of my way to make sure there was never a circumstance where a user requests something that doesn't exist.

Re: Python 3.11: “Zero cost” exception handling

#34

Post by formerly_proven » Wed, Oct 06, 2021, 2:36 PM UTC

bregma wrote: post #22

ziml77 wrote: post #2

They weren't zero cost before? In a language where idiomatic control flow uses exceptions? That's crazy! I've felt weird using exceptions like that but I always assumed that CPython was optimized to minimize overhead of exceptions and exception handlers.

Exceptions always have a cost. What happened with many C++ runtimes is that they moved the execution cost of exceptions almost entirely into the exception raising mechanism so that the path of execution that does not raise exceptions has no overhead due to exceptions. This was not the case with the Python runtime. The "zero cost" implementation for C++ exceptions in the case of ELF binaries means storing a bunch of s…

Excellent post! Python Bytecode is a little more naive / high level than this though, so stuff like exceptions and their nested handlers are actually implemented in the VM itself. The VM has essentially a second stack for exception and context manager blocks. The compiled bytecode essentially looks like this:

    SETUP_TRY 10 # address where exceptions will be handled
    some stuff that might explode
    POP_BLOCK
    JUMP 20  # jump over exception handlers
    LOAD exception type  # address 10
    COMPARE  # check if exception type matches
    ... handler for the type
    stuff after the try-except  # address 20

The corresponding source would look like

    try:
        some stuff that might explode
    except exception type:
         ... handler for the type
    stuff after the try-except

Python's bytecode compiler is generally a 1:1 translation of the AST; it never optimizes, e.g:

    [value for value in list]

Translates to something like

    LIST_NEW
    FOR_EACH
    STORE
    LOAD
    LIST_APPEND
    JUMP BACK

Note how "value" generated a store-load pair.

The VM checks whether PyErr (pointer to exception) is set after basically everything. Similarly, extension modules check PyErr after every call into the interpreter, e.g.

    PyObject *attr = PyObject_GetAttr(someattr, somepystr);
    if(PyErr_Occurred()) { // or !attr
        // handle exception
    }

This gets old pretty fast.

Re: Python 3.11: “Zero cost” exception handling

#35

Post by zohch » Wed, Oct 06, 2021, 2:41 PM UTC

ghshephard wrote: post #31

zohch wrote: post #3

Earlier quoted context omitted.

> I've felt weird using exceptions like that How should they be used instead? Maybe I don't understand what you mean by "idiomatic control flow uses exceptions" - could you give an example. Maybe there is some use of exceptions that I'm not quite familiar with in Python.

This covers it very well: https://devblogs.microsoft.com/python/idiomatic-python-eafp-... In particular, this is not idiomatic python: if "key" in dict_: value += dict_["key"] But this is: try: value += dict_["key"] except KeyError: pass I too hate using the exception handling in this way, and if you aren't careful, you end up papering over other unexpected exceptions in your code, so you have to be (A) very specific…

> In particular, this is not idiomatic python

According to this article, I think his case is rather weak. The conclusion does not seem to follow from the premise to me.

Re: Python 3.11: “Zero cost” exception handling

#36

Post by mywittyname » Wed, Oct 06, 2021, 2:54 PM UTC

ghshephard wrote: post #31

zohch wrote: post #3

Earlier quoted context omitted.

> I've felt weird using exceptions like that How should they be used instead? Maybe I don't understand what you mean by "idiomatic control flow uses exceptions" - could you give an example. Maybe there is some use of exceptions that I'm not quite familiar with in Python.

This covers it very well: https://devblogs.microsoft.com/python/idiomatic-python-eafp-... In particular, this is not idiomatic python: if "key" in dict_: value += dict_["key"] But this is: try: value += dict_["key"] except KeyError: pass I too hate using the exception handling in this way, and if you aren't careful, you end up papering over other unexpected exceptions in your code, so you have to be (A) very specific…

If you don't care about a key existing, then this works.

    do_something(dict_.get("key", None))

I use that a lot for data parsing. Passing the exception is not very clean IMHO. I stick to d[key] nomenclature when I need assurance that all the keys are present in the dictionary, and .get(key,None) when I don't.

Re: Python 3.11: “Zero cost” exception handling

#37

Post by SideburnsOfDoom » Wed, Oct 06, 2021, 2:56 PM UTC

pca006132 wrote: post #15

ziml77 wrote: post #2

They weren't zero cost before? In a language where idiomatic control flow uses exceptions? That's crazy! I've felt weird using exceptions like that but I always assumed that CPython was optimized to minimize overhead of exceptions and exception handlers.

Zero cost refers to the cost when no exception is thrown , not the overhead of exceptions. It may be more expensive throwing an exception under "zero cost" exception model, as throwing an exception may require parsing some data in the executable. (I'm not sure about the implementation, so this is just a may...)

> Zero cost refers to the cost when no exception is thrown, not the overhead of exceptions

There are recent benchmarks where .NET "zero cost when an try-catch block is present and exception not thrown" turned out to be significantly slower then the alternative without such a block.

It turns out that the try-catch block is a barrier across which some optimisations and re-organisations (e.g. method inlining) cannot happen, and that the compiler & JIT normally do a lot of these. So it might be "zero cost" but it might also prevent wins.

Edit: see here https://blogs.msmvps.com/peterritchie/2007/06/22/performance...

https://stackoverflow.com/questions/1308432/do-try-catch-blo...

Re: Python 3.11: “Zero cost” exception handling

#38

Post by stevesimmons » Wed, Oct 06, 2021, 3:00 PM UTC

ghshephard wrote: post #31

zohch wrote: post #3

Earlier quoted context omitted.

> I've felt weird using exceptions like that How should they be used instead? Maybe I don't understand what you mean by "idiomatic control flow uses exceptions" - could you give an example. Maybe there is some use of exceptions that I'm not quite familiar with in Python.

This covers it very well: https://devblogs.microsoft.com/python/idiomatic-python-eafp-... In particular, this is not idiomatic python: if "key" in dict_: value += dict_["key"] But this is: try: value += dict_["key"] except KeyError: pass I too hate using the exception handling in this way, and if you aren't careful, you end up papering over other unexpected exceptions in your code, so you have to be (A) very specific…

The difference in your "which of the two look better" example is:

(a) has one dict operation plus an exception which rarely occurs and is nearly zero cost if it doesn't.

versus

(b) has nearly always two dict operations, plus a possibly incorrect assumption that the dict will not be mutated between the "if key in dict" and "dict[key]" operations.

Re: Python 3.11: “Zero cost” exception handling

#39

Post by vlovich123 » Wed, Oct 06, 2021, 3:03 PM UTC

zeotroph wrote: post #30

bregma wrote: post #22

Earlier quoted context omitted.

Exceptions always have a cost. What happened with many C++ runtimes is that they moved the execution cost of exceptions almost entirely into the exception raising mechanism so that the path of execution that does not raise exceptions has no overhead due to exceptions. This was not the case with the Python runtime. The "zero cost" implementation for C++ exceptions in the case of ELF binaries means storing a bunch of s…

Note that the generated instructions - even if they are never run as in [1] - still consume cache and might hinder further optimizations. Which is why `noexcept` is becoming more popular. And because there is no GC, code must be written to be exception-safe in all conditions which is often forgotten. 1: https://godbolt.org/z/bKfG14P64 - the difference between `e()` and `n()` is that one is marked `noexcept`. Both `f(…

Isn’t exception handling code placed in the cold section of the binary these days so that the impact on cache is nonexistent?

Yup, the generated assembly does this. There’s some minimal extra instructions still to setup the frame but the bulk of the exception handling lives elsewhere.

Re: Python 3.11: “Zero cost” exception handling

#40

Post by bregma » Wed, Oct 06, 2021, 3:03 PM UTC

zeotroph wrote: post #30

bregma wrote: post #22

Earlier quoted context omitted.

Exceptions always have a cost. What happened with many C++ runtimes is that they moved the execution cost of exceptions almost entirely into the exception raising mechanism so that the path of execution that does not raise exceptions has no overhead due to exceptions. This was not the case with the Python runtime. The "zero cost" implementation for C++ exceptions in the case of ELF binaries means storing a bunch of s…

Note that the generated instructions - even if they are never run as in [1] - still consume cache and might hinder further optimizations. Which is why `noexcept` is becoming more popular. And because there is no GC, code must be written to be exception-safe in all conditions which is often forgotten. 1: https://godbolt.org/z/bKfG14P64 - the difference between `e()` and `n()` is that one is marked `noexcept`. Both `f(…

Yeah, noexcept(true) is identical to wrapping the function with a try-catch construct in which the catch clause simply calls std::terminate(). Your godbolt example doesn't include the definition of n() so it doesn't show that. Adding noexcept(true) has a cost (because it's a kind of exception handling) but also allows the compiler to optimize out some of that cost under some circumstances.

Nevertheless, an example that demonstrates the exception path doesn't say much about the "zero-cost" non-exception path. In C++ exception handling is expensive. Outside of actually directly handling exceptions they have zero cost. There are no generated instructions. They do not consume cache. They do not hinder further optimizations.