Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Log level 'error' should mean that something needs to be fixed

utcc.utoronto.ca

Post reply on HN
Print view View original

191–200 of 313 posts

Re: Log level 'error' should mean that something needs to be fixed

#191

Post by ivan_gammel »

dpark wrote: post #190
ivan_gammel wrote: post #184

Earlier quoted context omitted.

> almost always just maps to logging anyway Yes, that’s the point. You log it until you encounter it for the first time, then you know more and can do something meaningful. E.g. let’s say you build an API client and library offers callback for HTTP 429. You don’t expect it to happen, so just log the errors in a generic handler in client code, but then after some business logic change you hit 429 for the first time. I…

Defining a callback for every situation where a library might encounter an unexpected condition and pointing them all at the logs seems like a massive waste of time. I would much prefer a library have sane defaults, reasonable logging, and a way for me to plug in callbacks where needed. Writing On429 and a hundred other functions that just point to Logger.Log is not a good use of time.

This sub-thread in my understanding is about a special case (a non-error mode that client may want to avoid, in which case explicit callback makes sense), not about all possible unexpected errors. I’m not suggesting hooks as the best approach. And of course “on429” is the last thing I would think about when designing this. There are better ways.

Re: Log level 'error' should mean that something needs to be fixed

#192

Post by dpark »

ivan_gammel wrote: post #191
dpark wrote: post #190

Earlier quoted context omitted.

Defining a callback for every situation where a library might encounter an unexpected condition and pointing them all at the logs seems like a massive waste of time. I would much prefer a library have sane defaults, reasonable logging, and a way for me to plug in callbacks where needed. Writing On429 and a hundred other functions that just point to Logger.Log is not a good use of time.

This sub-thread in my understanding is about a special case (a non-error mode that client may want to avoid, in which case explicit callback makes sense), not about all possible unexpected errors. I’m not suggesting hooks as the best approach. And of course “on429” is the last thing I would think about when designing this. There are better ways.

If the statement is just that sometimes it’s appropriate to have callbacks, absolutely. A library that only logs in places where it really needs a callback is poorly designed.

I still don’t want to have to provide a 429 callback just to log, though. The library should log by default if the callback isn’t registered.

Re: Log level 'error' should mean that something needs to be fixed

#193

Post by lanstin »

ivan_gammel wrote: post #78
layer8 wrote: post #76

> When implementing logging, it's important to distinguish between an error from the perspective of an individual operation and an error from the perspective of the overall program or system. Individual operations may well experience errors that are not error level log events for the overall program. You could say that an operation error is anything that prevents an operation from completing successfully, while a pro…

Libraries should not log on levels above DEBUG, period. If there’s something worthy for reporting on higher levels, pass this information to client code, either as an event, or as an exception or error code.

I have a logging level I call "log lots" where it will log the first time with probability 1, but as it hits more often the same line, it will log with lower and lower probability bottoming out around 1/20000 times. Sort of a "log with probability proportional to the unlikiness of the event". So if I get e.g. sporadic failures to some back end, I will see them all, but if it goes down hard I will see it is still down but also be able to read other log msgs.

Re: Log level 'error' should mean that something needs to be fixed

#194

Post by pca006132 »

Too wrote: post #92
layer8 wrote: post #76

> When implementing logging, it's important to distinguish between an error from the perspective of an individual operation and an error from the perspective of the overall program or system. Individual operations may well experience errors that are not error level log events for the overall program. You could say that an operation error is anything that prevents an operation from completing successfully, while a pro…

This is why it’s almost always wrong for library functions to log anything, even on ”errors”. Pass the status up through return values or exceptions. As a library author you have no clue as how an application might use it. Multi threading, retry loops and expected failures will turn what’s a significant event in one context into what’s not even worthy of a debug log in another. No rule without exceptions of course, o…

Wonder if someone used effect handlers for error logging. Sounds like a natural and modular way of handling this problem.

Re: Log level 'error' should mean that something needs to be fixed

#195

Post by lanstin »

arwhatever wrote: post #99
eterm wrote: post #45

How I'd personally like to treat them: - Critical / Fatal: Unrecoverable without human intervention, someone needs to get out of bed, now. - Error : Recoverable without human intervention, but not without data / state loss. Must be fixed asap. An assumption didn't hold. - Warning: Recoverable without intervention. Must have an issue created and prioritised. ( If business as usual, this could be downgrading to INFO. )…

I like to think of “warning” as something to alert on statistically, e.g. incorrect password attempt rate jumps from 0.4% of login attempts to 99%.

This point is important - the value of a log is inextricably tied to its unlikelihood. Which depends on so many things in the context.

Re: Log level 'error' should mean that something needs to be fixed

#196

Post by ivan_gammel »

dpark wrote: post #192
ivan_gammel wrote: post #191

Earlier quoted context omitted.

This sub-thread in my understanding is about a special case (a non-error mode that client may want to avoid, in which case explicit callback makes sense), not about all possible unexpected errors. I’m not suggesting hooks as the best approach. And of course “on429” is the last thing I would think about when designing this. There are better ways.

If the statement is just that sometimes it’s appropriate to have callbacks, absolutely. A library that only logs in places where it really needs a callback is poorly designed. I still don’t want to have to provide a 429 callback just to log, though. The library should log by default if the callback isn’t registered.

It doesn’t have to provide a specific callback. This can be a starting point (Java):

  var client = aClient()
   .onError((request,response) -> { 
      LOG.debug(…); 
      return FAIL; 
   }).build();
And eventually you do this:

   return switch(response.code()) {
      case 429 -> RETRY;
      default -> FAIL;
   }
Or something more interesting, e.g. with more details of retry strategy.

Re: Log level 'error' should mean that something needs to be fixed

#197

Post by cluckindan »

TZubiri wrote: post #139
cogman10 wrote: post #132

Earlier quoted context omitted.

Depending on the language and logging framework, debug/trace logging can be acceptable in a library. But you have to be extra careful to make sure that it's ultimately a no-op. A common problem in Java is someone will drop a log that looks something like this `log.trace("Doing " + foo + " to " + bar);` The problem is, especially in a hot loop, that throw away string concatenation can ultimately be a performance probl…

How about wrapping the log.trace param in a lambda and monkeypatching log.trace to take a function that returns a string, and of course pushing the conditional to the monkeypatched func.

Then you still have the overhead of the log.trace function call and the lambda construction (which is not cheap because it has closure over the params being logged and is passed as a param to a function call, so probably gets allocated on the heap)

Re: Log level 'error' should mean that something needs to be fixed

#198

Post by lanstin »

DanHulton wrote: post #50
colechristensen wrote: post #43

Earlier quoted context omitted.

How about this: - An error is an event that someone should act on. Not necessarily you. But if it's not an event that ever needs the attention of a person then the severity is less than an error. Examples: Invalid credentials. HTTP 404 - Not Found, HTTP 403 Forbidden, (all of the HTTP 400s, by definition) It's not my problem as a site owner if one of my users entered the wrong URL or typed their password wrong, but i…

If you're logging and reporting on ERRORs for 400s, then your error triage log is going to be full of things like a user entering a password with insufficient complexity or trying to sign up with an email address that already exists in your system. Some of these things can be ameliorated with well-behaved UI code, but a lot cannot, and if your primary product is the API, then you're just going to have scads of ERRORs…

The frequency is important and so is the answer to "could we have done something different ourselves to make the request work". For example in credit card processing, if the remote network declines, then at first it seems like not your problem. But then it turns out for many BINs there are multiple choices for processing and you could add dynamic routing when one back end starts declining more than normal. Not a 5xx and not a fault in your process, but a chance to make your customer experience better.

Re: Log level 'error' should mean that something needs to be fixed

#199

Post by lanstin »

rwmj wrote: post #21

And the second rule is make all your error messages actionable . By that I mean it should tell me what action to take to fix the error (even if that action means hard work, tell me what I have to do).

Also put the fucking data in the message that led to the decision to emit the logs. I can't remember how many times I have had a three part test trigger a log "blah: called with illegal parameters, shouldn't happen" and the illegal parameters were not logged.

Re: Log level 'error' should mean that something needs to be fixed

#200

Post by aunty_helen »

Good logging is critical and actually having the logs turned on in production. No point writing logs if you silence them.

My company now has a log aggregator that scans the logs for errors, when it finds one, creates a Trello card, uses opus to fix the issue and then propose a PR against the card. These then get reviewed, finished if tweaks are necessary and merged if appropriate.

Post reply on HN