Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python exceptions considered an anti-pattern

sobolevn.me

Post reply on HN
Print view View original

21–30 of 69 posts

Re: Python exceptions considered an anti-pattern

#21

Post by wvenable »

Glench wrote: post #9

I'm glad this issue is getting some attention. I found the "Exceptions are not exceptional" section to capture something I've noticed several times — that code in Python can fail at a huge number of points even in a small function, nevermind the many exceptions that might happen in nested functions, or code we're using from libraries. We should have language-level mechanisms for being explicit about what's supposed t…

I disagree. The locations of the failure don't matter. All these returning errors and propagating them manually is just masturbation. If an error occurs, you only care about where you can restart the entire transaction (whether that be re-issue a network request, re-try writing a file, etc). That might only be in a handful of places in your code; so that is where you put exception handlers. It literally doesn't matter which of the thousands of possible methods up the call stack threw an exception. So why are suddenly so concerned with tracking that everywhere?

If I divide by zero, that's probably just a bug in my program. Why am I wasting time dealing with that on every single division expression everywhere? It makes no sense.

Re: Python exceptions considered an anti-pattern

#22

Post by mindslight »

It amazes me how enduring formulaic it is to single out some particular design tradeoff of a language, draw up some examples of expressing something where that tradeoff creates worse code, and then act like it's some mortal flaw in the language.

Python chose untyped exceptions, period. How is this surprising, given that its basis is untyped parameters?

If you don't like that, use Java with its checked exceptions. Or remove exceptions from the implicit monad altogether and use C or Rust. Just don't then go on to write some lengthy post about how any one of those is too explicit.

Re: Python exceptions considered an anti-pattern

#26

Post by kissgyorgy »

I never understood, that people who don't understand Python at all, or don't like essential Python constructs, why they bother using it? Why not use a different language in the first place? If you are using this library, you are not writing Python anymore and you lost the biggest advantage of the language: simplicity.

Re: Python exceptions considered an anti-pattern

#27

Post by futureastronaut »

StavrosK wrote: post #25
futureastronaut wrote: post #20

Earlier quoted context omitted.

Maybe you should be familiar with it.

Cool, I'll file it under "to maybe be familiar with, things", thanks.

Or mention "PTSD" when somebody brings up Java to signal that you're above such foolishness, and rely on nobody seeking an explanation for your aversion.

Re: Python exceptions considered an anti-pattern

#28

Post by ltbarcly3 »

Lets all agree that it is impossible to write code that will never have an unexpected outcome. Imagine that we somehow write a function that is totally bullet-proof. It can't fail, it will always do precisely what it was intended to do. Further, lets say whenever we run this function we run it on N computers and take the consensus result if any of the computers disagree. No matter how large N is, if we run the function enough times eventually we will get a majority of the computers to agree, return the same result, and that result will be wrong. Whether it's from cosmic rays resetting bits in memory, or multiple cosmic ray strikes resetting multiple bits and thus defeating ECC, sooner or later things will break no matter what you do. Even if you shield all the computers with 5 meters of pre-nuclear age lead, eventually it will break.

The point is that it is just not possible to get to perfect reliability. Your actual reliability is always going to be less than 100%. You can invest money and effort to get closer to 100%, but obviously you are going to get diminishing returns.

The correct analysis is to decide where the optimal trade-off is between investing in reliability and the return on that investment.

Example 1: You are calling a web service that checks the weather. The service might be down. If it is down you wait a few seconds and try it again. The 'cost' of it being down is that a user doesn't see the current weather. Is it worthwhile to carefully try to determine whether the error when calling the service is due to a server returning a 500 status code versus invalid json?

No, it's not worth it. Either way the client can't use the response. In fact, it doesn't matter what causes the exception, since anything that goes wrong can't be corrected by the client. Whether it's bad json, a network failure, dns failure, the server is being rebooted, the webserver is misconfigured, or the device is in airplane mode, the resolution is always the same, wait and try again in a few seconds. Exceptions work pretty much ideally in this case, you only have to code the 'happy' path and handle all exceptions the same way generically.

Example 2: You are writing code to update a database containing financial transactions. If something goes wrong in an unexpected way you need to make sure the financial data isn't updated or left partially updated.

Again, you don't care about unexpected exceptions. For failures you expect and are coding to work around them, possibly by catching the generic exception where it happens deep in the call stack, and then raising your own exception class which properly identifies the error and contains the context necessary to perform the recovery. For example, if you need to send an email via receipt for the transaction, you call some function which formats and sends the email. That function fails due to the email server being unreachable. The network exception is caught and an EmailCantBeSent exception is raised with the relevant details in it (the user_id you were emailing, the transaction_id the email is for). The resolution is to log a critical error and insert a record to the database with the relevant details of the email so that someone can make sure it is sent later. Then it continues with the transaction. If something unexpected happens the database transaction is never committed.

My point is that there are two kinds of exceptions you run into, the ones you are being careful to trap and resolve as part of your applications design, and the ones that you aren't trying to resolve and so result in just a generic 'this failed' situation.

So finally getting back to finding the optimal tradeoff between investment to improve reliability and payback on that investment, you just need to make sure your generic failures are rare enough that you aren't pushed far from that optimal point, which is almost always going to be the case, even if you basically don't ever handle any exceptions and only code for the happy path. Obviously there are tons of counter-examples and sometimes you need to make sure things work even when something goes wrong (if you are working on an autopilot you will require much higher reliability and so much more careful planning to reach it compared to a twitter client, where you just need to not lose what the person typed).

Ok that's a lot longer than I intended.

TLDR; If you do any kind of analysis on why code fails and what you should do about that, you quickly realize that this library doesn't help at all. This library isn't even bad, the problem it is meant to solve is not well posed.

Re: Python exceptions considered an anti-pattern

#29

Post by kstrauser »

chillacy wrote: post #8

Counterargument: Exceptions are Pythonic https://jeffknupp.com/blog/2013/02/06/write-cleaner-python-u...

I'm soundly in that camp. For instance, we write API endpoints that look a lot like this:

  def update_password_view(session_cookie):
      values = request_params(['old_password', 'new_password'])
      user = get_user(session_cookie)
      verify_password(user, values['old_password'])
      update_password(user, values['new_password'])
      return 200
  
  def request_params(param_names):
      values = {}
      for key in param in param_names:
          try:
              values[key] = request.params[key]
          except KeyError:
              raise BadRequestError('Missing parameter', key)
      return values
  
  def get_user(session_cookie):
      users = db.get_user_with_session(session_cookie)
      if len(users) != 1:
          raise NotFoundError('No user with that session')
  
      return users[0]
  
  def verify_password(user, old_password):
      if hash(user.old_password) != hash(old_password):
          raise BadRequestError('Bad password')
  
  def update_password(user, new_password):
      user.password = hash(new_password)
      db.update_user(user)
Notice that each function raises an HTTP-ready exception, so update_password_view has no explicit error handling of its own. You can look at that function and read the intent of how it actually works, as each line is only reachable if the one before it 100% succeeded. After `user = get_user(...)`, you know that `user` will have valid data and not some sentinel value you have to check for.

Our actual implementations are more subtle. We have our own exception hierarchy with classes like `UserNotFoundError` or `BadPasswordError` that subclass the corresponding HTTP error classes, so you can still write code like:

  def upsert_user(data):
      try:
          user = get_user_by_email(data.email_address)
      except UserNotFoundError:
          user = User(data)
          db.save(user)
      do_something_with(user)
in the cases where that exception isn't fatal.

In practice, we've found this coding style to be much easier maintain than idioms like `if not_found(user): return None` where you spend half your lines of code explicitly checking return values for error sentinels. Life's too short to live like that.

Re: Python exceptions considered an anti-pattern

#30

Post by rpedela »

kissgyorgy wrote: post #26

I never understood, that people who don't understand Python at all, or don't like essential Python constructs, why they bother using it? Why not use a different language in the first place? If you are using this library, you are not writing Python anymore and you lost the biggest advantage of the language: simplicity.

Often the language is chosen because it is the only option like JS for web, or it is the best option for the project. For example, you are doing physics and you need a general purpose language that is easy to write and handles gigantic numbers correctly so you pick Python.
Post reply on HN