Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python 3.11: “Zero cost” exception handling

bugs.python.org

Post reply on HN
Print view View original

61–70 of 96 posts

Re: Python 3.11: “Zero cost” exception handling

#61

Post by notdonspaulding »

_bohm wrote: post #23
brianwawok wrote: post #11

Earlier quoted context omitted.

Right, but the better way to actually write this is something like entry = Entry.objects.filter(blog__id=1, entry_number=1).first() if entry is None: # deal with does not exist Maybe it's my scala/Java background shining through, but we are big Django users and we ban the "catch exceptions as standard" workflow, because there is almost always a cleaner way...

This fails to raise an error if there is more than one object matching the given filters

Presumably entry_number is unique_together with blog_id. Otherwise the original code is also not handling the MultipleObjectsReturned exception.

Generally speaking, I tend toward the cleanest code being:

    blog = Blog.objects.get(id=1)
    entry = blog.entry_set.filter(entry_number=1).first()
    if entry is None:
        handle_missing_entry()
    handle_entry(entry)
But it does suffer from having the extra DB query in there, which may or may not be helpful, depending on the surrounding code (and whether or not you'll be using the blog instance anywhere else).

Re: Python 3.11: “Zero cost” exception handling

#62

Post by carapace »

albertzeyer wrote: post #45
bregma wrote: post #22

Earlier quoted context omitted.

Exceptions always have a cost. What happened with many C++ runtimes is that they moved the execution cost of exceptions almost entirely into the exception raising mechanism so that the path of execution that does not raise exceptions has no overhead due to exceptions. This was not the case with the Python runtime. The "zero cost" implementation for C++ exceptions in the case of ELF binaries means storing a bunch of s…

> exceptions are a fundamental control-flow mechanism in Python Where do you have this from? In almost all Python code I have seen, exceptions are still for, as the name says, exceptions. So in the common cases, there would be no exceptions raised. I would assume that the exception handling code (under `except ...`) will be run only a tiny fraction of times compared to the other code, at least in most cases. I would…

In Python e.g. instead of doing this:

    if key in some_dict:
        foo = some_dict[key]
        ... # A
    else:
        ... # B
you would do this:

    try:
        foo = some_dict[key]
    except KeyError:
        ... # B
    else:
        ... # A

Re: Python 3.11: “Zero cost” exception handling

#63

Post by pythonb3sss »

olah_1 wrote: post #49
pythonb3sss wrote: post #47

Wow! Prior to reading this, I was not aware of "Zero Cost" exception handling. While I am only a Python developer, I always assumed that in any programming language, exception handling, regardless of whether an exception is raised or not, cost some CPU cycles. I work at an HFT firm and they test their changes in equations in Python programs on crypto rather than C++. So I resorted to using try-except blocks in Python…

You may enjoy programming in Elixir if you like that style. In Elixir, you only program the “happy path” and just let things fail. Then you rely on supervisor processes to handle the exceptions/errors. Well, at least that is the idea. I think people still do tests and function guards and things. but the “let it fail” idea is definitely part of the Erlang/Elixir world. The sad thing is that there really isn’t any “lea…

Well, I don't enjoy the "happy path" programming. Admittedly, this implementation to improve speed feels a bit hacky. I only did it because it had a measurable impact on the computational performance of my program. In my other grunt worker scripts, I actually prefer if-elif-else statements because they make code readability better for other programmers who are not Python "natives", but use the scripts or modify them to suit their use cases.

Re: Python 3.11: “Zero cost” exception handling

#64

Post by Lendal »

pythonb3sss wrote: post #47

Wow! Prior to reading this, I was not aware of "Zero Cost" exception handling. While I am only a Python developer, I always assumed that in any programming language, exception handling, regardless of whether an exception is raised or not, cost some CPU cycles. I work at an HFT firm and they test their changes in equations in Python programs on crypto rather than C++. So I resorted to using try-except blocks in Python…

For me it isn't about the cost. Modern languages like Go and Rust separate the error handling from the conventional logic, and that makes the code more readable. It's my only complain about Python, (outside of performance of course). In Python when you see a `try`, you don't know if it's because there's error handling going on, or if it's because that's the only way to achieve a certain goal due to Python being designed to mingle logic with error handling. After doing projects in Go and Rust, I can see the value in separating the two, and that makes me sad that Python is old now.

Maybe what they're planning to do with this is allow wrappers to hide the places where exception handling is gratuitous, and therefore try to bring Python forward into the world of more modern languages.

Re: Python 3.11: “Zero cost” exception handling

#65

Post by louissan »

ubercore wrote: post #4
zohch wrote: post #3

Earlier quoted context omitted.

> I've felt weird using exceptions like that How should they be used instead? Maybe I don't understand what you mean by "idiomatic control flow uses exceptions" - could you give an example. Maybe there is some use of exceptions that I'm not quite familiar with in Python.

The first that came to mind is how `get()` is handled in Django's ORM. The idiomatic way to look for a single object is to use `get`, then catch a `DoesNotExist` exception: From https://docs.djangoproject.com/en/3.2/ref/models/querysets/#... from django.core.exceptions import ObjectDoesNotExist try: blog = Blog.objects.get(id=1) entry = Entry.objects.get(blog=blog, entry_number=1) except ObjectDoesNotExist: print("Ei…

Or even?:

try: blog = Blog.objects.get(slug__icontains='some text') except Blog.ObjectDoesNotExist: print("No blog could be found") except Blog.MultipleObjectsReturned: print('More than one blog!')

Re: Python 3.11: “Zero cost” exception handling

#66

Post by alex_smart »

masklinn wrote: post #9
ziml77 wrote: post #2

They weren't zero cost before? In a language where idiomatic control flow uses exceptions? That's crazy! I've felt weird using exceptions like that but I always assumed that CPython was optimized to minimize overhead of exceptions and exception handlers.

> In a language where idiomatic control flow uses exceptions? That's crazy! Seems like the opposite: if exceptions are extremely rare then you want to optimise the case where they’re not raised, at the cost of the other one. If exceptions are common then it matters a lot less, you may even want to avoid 0ce depending on the impact on the raised case.

I don't think python has the mindset of "exceptions are extremely rare". That is probably what the OC meant by python being "a language where idiomatic control flow uses exceptions". As an example, every iterator in python signals its end by throwing a StopIteration exception. So, every "for x in iter" has the interpreter throwing and catching an exception.

Re: Python 3.11: “Zero cost” exception handling

#67

Post by sonthonax »

bregma wrote: post #22
ziml77 wrote: post #2

They weren't zero cost before? In a language where idiomatic control flow uses exceptions? That's crazy! I've felt weird using exceptions like that but I always assumed that CPython was optimized to minimize overhead of exceptions and exception handlers.

Exceptions always have a cost. What happened with many C++ runtimes is that they moved the execution cost of exceptions almost entirely into the exception raising mechanism so that the path of execution that does not raise exceptions has no overhead due to exceptions. This was not the case with the Python runtime. The "zero cost" implementation for C++ exceptions in the case of ELF binaries means storing a bunch of s…

What’s a DSO?

Re: Python 3.11: “Zero cost” exception handling

#68

Post by pythonb3sss »

klodolph wrote: post #48
pythonb3sss wrote: post #47

Wow! Prior to reading this, I was not aware of "Zero Cost" exception handling. While I am only a Python developer, I always assumed that in any programming language, exception handling, regardless of whether an exception is raised or not, cost some CPU cycles. I work at an HFT firm and they test their changes in equations in Python programs on crypto rather than C++. So I resorted to using try-except blocks in Python…

There are some weird performance optimizations in Python, e.g., item = some_dict.get(key) if item is None: # key does not exist Versus try: item = some_dict[key] except KeyError: # key does not exist When I tested these (admittedly, a while ago), which one was faster depended on how often the key was missing. If “missing key” was an expected case, the first one was faster. If “missing key” was uncommon, the second wa…

I am going to speculate here, so if I'm wrong please point it out.

Here, the number of steps directly affect the time.

In the first approach, the ".get()" method first analyses the type of "some_dict" and then uses an internal variable (the ones surrounded by double underscores) to try and fetch the value by using the provided key. If the key is present, then the value is returned, if not then a default value is returned. So if the key does not exist, the returning the default value saves 1 step (that of fetching the value from the map)

In the second approach, the exception raises the number of steps because the type of error has to be determined and the stack is traced every time an exception is raised. So the more exceptions are raised, the slower the code gets.

I tested this with 3.9.7 right now and in my testing, the runtime of first approach was virtually unchanged, while the second one was faster if exceptions were raised ~12% of the time or less. (I ran both 10 million times)

Re: Python 3.11: “Zero cost” exception handling

#69

Post by mjw1007 »

sonthonax wrote: post #67
bregma wrote: post #22

Earlier quoted context omitted.

Exceptions always have a cost. What happened with many C++ runtimes is that they moved the execution cost of exceptions almost entirely into the exception raising mechanism so that the path of execution that does not raise exceptions has no overhead due to exceptions. This was not the case with the Python runtime. The "zero cost" implementation for C++ exceptions in the case of ELF binaries means storing a bunch of s…

What’s a DSO?

Dynamic Shared Object (in this context, probably a .so file).

Re: Python 3.11: “Zero cost” exception handling

#70

Post by ok123456 »

cnband wrote: post #21

The tenacity of people getting excited over micro optimizations in Python for more than two decades is remarkable. Nothing has happened despite monumental speed programs that were broadly advertised and marketed to corporations. Meanwhile, SBCL has an industrial strength compiler that predates Python and its trademark (the SBCL compiler was called "Python" before the trademark, thereby invalidating it). Python (the l…

It's when you put all these small optimizations together that it leads to something remarkable. It's analogous to video codecs: there are a bunch of individual optimizations that alone don't look that impressive, only saving ~1% here or there. But once they all are working together, you see savings of 10-50%.
Post reply on HN