Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Log level 'error' should mean that something needs to be fixed

utcc.utoronto.ca

Post reply on HN
Print view View original

141–150 of 313 posts

Re: Log level 'error' should mean that something needs to be fixed

#141

Post by Izkata »

echelon wrote: post #94
Too wrote: post #92

Earlier quoted context omitted.

This is why it’s almost always wrong for library functions to log anything, even on ”errors”. Pass the status up through return values or exceptions. As a library author you have no clue as how an application might use it. Multi threading, retry loops and expected failures will turn what’s a significant event in one context into what’s not even worthy of a debug log in another. No rule without exceptions of course, o…

You need a tuple: (context, level) The application owner should be able to adjust the contexts up or down. This is the point of ownership and where responsibility over which logs matter is handled. A library author might have ideas and provide useful suggestions, but it's ultimately the application owner who decides. Some libraries have huge blast radius and their `error` might be your `error` too. In other contexts,…

Python's built-in logging is the same if used correctly, where the library gets a logger based on its module name (this part isn't enforced) and the application can add a handler to that logger to route the logs differently if needed.

Re: Log level 'error' should mean that something needs to be fixed

#142

Post by TZubiri »

esrauch wrote: post #118
Too wrote: post #92

Earlier quoted context omitted.

This is why it’s almost always wrong for library functions to log anything, even on ”errors”. Pass the status up through return values or exceptions. As a library author you have no clue as how an application might use it. Multi threading, retry loops and expected failures will turn what’s a significant event in one context into what’s not even worthy of a debug log in another. No rule without exceptions of course, o…

I think an example where libraries could sensibly log error is if you have a condition which is recoverable but may cause a significant slowdown, including a potential DoS issue, and the application owner can remediate. You don't want to throw because destroying someone's production isn't worth it. You don't want to silent continue in that state because realistically there's no way for application owner to understand…

We call those warnings, and it's very common to downgrade errors to warnings by wrapping an exception and printing the trace as you would an exception.

Re: Log level 'error' should mean that something needs to be fixed

#144

Post by ivan_gammel »

layer8 wrote: post #102
ivan_gammel wrote: post #86

Earlier quoted context omitted.

The main difference is that library is not aware of the context of the execution of the code, so cannot decide, whether the problem is expected, recoverable or severe.

The same is true for programs that are being invoked. The program only knows relative to its own purpose, and the same is again true for libraries. I don’t see the difference, other than, as already mentioned, the mechanism of program vs. library invocation. Consider a Smalltalk-like system, or something like TCL, that doesn’t distinguish between programs and libraries regarding invocation mechanism. How would you ha…

The purpose of a program and of a library is different and intent of the authors of the code is usually clear enough to make the distinction in context. Small composable programs aren’t interesting case here, they shouldn’t be verbose anyway even to justify multiple logging levels (it’s probably just set to on/off using a command line argument).

Re: Log level 'error' should mean that something needs to be fixed

#145

Post by TZubiri »

hinkley wrote: post #82
layer8 wrote: post #76

> When implementing logging, it's important to distinguish between an error from the perspective of an individual operation and an error from the perspective of the overall program or system. Individual operations may well experience errors that are not error level log events for the overall program. You could say that an operation error is anything that prevents an operation from completing successfully, while a pro…

Log4j has the ability to filter log levels by subject matter for twenty years. In Java you end up having to use that a lot for this reason.

Oh that library that gives you a write() wrapper in exchange for RCE vulns

Re: Log level 'error' should mean that something needs to be fixed

#146

Post by raldi »

plorkyeran wrote: post #134
raldi wrote: post #74

Earlier quoted context omitted.

Exactly: When you're building software, it has lots of defects (and, thus, error logging). When it's mature, it should have few defects, and thus few error logs, and each one that remains is a bug that should be fixed.

I don't understand why you seem to think you're disagreeing with the article? If you're producing a lot of error logs because you have bugs that you need to fix then you aren't violating the rule that an error log should mean that something needs to be fixed.

I couldn’t agree more with the article. What made you think I disagreed?

Re: Log level 'error' should mean that something needs to be fixed

#147

Post by renewiltord »

Too wrote: post #92
layer8 wrote: post #76

> When implementing logging, it's important to distinguish between an error from the perspective of an individual operation and an error from the perspective of the overall program or system. Individual operations may well experience errors that are not error level log events for the overall program. You could say that an operation error is anything that prevents an operation from completing successfully, while a pro…

This is why it’s almost always wrong for library functions to log anything, even on ”errors”. Pass the status up through return values or exceptions. As a library author you have no clue as how an application might use it. Multi threading, retry loops and expected failures will turn what’s a significant event in one context into what’s not even worthy of a debug log in another. No rule without exceptions of course, o…

Conflicting goals for the predominant libraries is what causes this. Log4J2 has a rewrite appender that solves the problem. But if you want zero-copy etc I don’t think there’s such a solution.

Re: Log level 'error' should mean that something needs to be fixed

#148

Post by ivan_gammel »

TZubiri wrote: post #145
hinkley wrote: post #82

Earlier quoted context omitted.

Log4j has the ability to filter log levels by subject matter for twenty years. In Java you end up having to use that a lot for this reason.

Oh that library that gives you a write() wrapper in exchange for RCE vulns

Log4j is basically a design pattern. If you don’t like the library, Slf4j/logback are based on the same principles.

Re: Log level 'error' should mean that something needs to be fixed

#149

Post by HendrikHensen »

layer8 wrote: post #76

> When implementing logging, it's important to distinguish between an error from the perspective of an individual operation and an error from the perspective of the overall program or system. Individual operations may well experience errors that are not error level log events for the overall program. You could say that an operation error is anything that prevents an operation from completing successfully, while a pro…

> Should only “top-level” code ever log an error? That can make it difficult to identify the low-level root causes of a top-level failure.

Some languages (e.g. Java) include a stack trace when reporting an error, which is extremely useful when logging the error. It shows at exactly which point in the code the error was generated, and what the full call stack was to get there.

It's a real shame that "modern" languages or "low level" languages (e.g. Go, Rust) don't include this out of the box, it makes troubleshooting errors in production much more difficult, for exactly the reason you mention.

Re: Log level 'error' should mean that something needs to be fixed

#150

Post by msteffen »

layer8 wrote: post #102
ivan_gammel wrote: post #86

Earlier quoted context omitted.

The main difference is that library is not aware of the context of the execution of the code, so cannot decide, whether the problem is expected, recoverable or severe.

The same is true for programs that are being invoked. The program only knows relative to its own purpose, and the same is again true for libraries. I don’t see the difference, other than, as already mentioned, the mechanism of program vs. library invocation. Consider a Smalltalk-like system, or something like TCL, that doesn’t distinguish between programs and libraries regarding invocation mechanism. How would you ha…

Okay, but…most programs are written in Python or Rust or something, where invoking library functions is a lot safer, more ergonomic, more performant, and more common than spawning a subprocess and executing a program in it. Like you can’t really ignore the human expectations and conventions that are brought to bear when your code is run (the accommodation of which is arguably most of the purpose of programming languages).

When you publish a library, people are going to use it more liberally and in a wider range of contexts (which are therefore harder to predict, including whether a given violation requires human intervention)

Post reply on HN