Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Log level 'error' should mean that something needs to be fixed

utcc.utoronto.ca

Post reply on HN
Print view View original

111–120 of 313 posts

Re: Log level 'error' should mean that something needs to be fixed

#111

Post by dminuoso »

greatgib wrote: post #28
mfuzzey wrote: post #8

I think it's difficult to say without knowing how the system is deployed and administered. "If a SMTP mailer trying to send email to somewhere logs 'cannot contact port 25 on ', that is not an error in the local system" Maybe or maybe not. If the connection problem is really due to the remote host then that's not the problem of the sender. But maybe the local network interface is down, maybe there's a local firewall…

The point is that if your program itself take note of the error from the library it is ok. You, as the program owner, can decide what to do with it (error log or not). But if you are the SMTP library and that you unilaterally log that as an error. That is an issue.

This would require a complete new ecosystem and likely new language where any degradation of code flow becomes communicatable in a standardized and fully documented fashion.

The closest we have is something like Java with exceptions in type signatures, but we would have to ban any kind of exception capture except from final programs, and promote basically any logger call int an exception that you could remotely suppress.

We could philosophize about a world with compilers made out of unobtanium - but in this reality a library author cannot know what conditions are fixable or necessitate a fix or not. And structured logging lacks has way too many deficiencies to make it work from that angle.

Re: Log level 'error' should mean that something needs to be fixed

#112

Post by dnautics »

raldi wrote: post #75
ImPostingOnHN wrote: post #32

Earlier quoted context omitted.

That is a lagging indicator. By the time you're alerted, you've already failed by letting users experience an issue.

What alternative would you propose? Page the oncall whenever there's a single query timeout?

the alternative i propose is have deep understanding of your system before popping off with dumb one size fits all rules that don't make sense.

Re: Log level 'error' should mean that something needs to be fixed

#113

Post by hrimfaxi »

layer8 wrote: post #102
ivan_gammel wrote: post #86

Earlier quoted context omitted.

The main difference is that library is not aware of the context of the execution of the code, so cannot decide, whether the problem is expected, recoverable or severe.

The same is true for programs that are being invoked. The program only knows relative to its own purpose, and the same is again true for libraries. I don’t see the difference, other than, as already mentioned, the mechanism of program vs. library invocation. Consider a Smalltalk-like system, or something like TCL, that doesn’t distinguish between programs and libraries regarding invocation mechanism. How would you ha…

The mechanism of invocation is important. Most programs allow you to set the logging verbosity at invocation. Libraries may provide an interface to do so but their entry points tend to be more numerous.

Re: Log level 'error' should mean that something needs to be fixed

#115

Post by PartiallyTyped »

hinkley wrote: post #82
layer8 wrote: post #76

> When implementing logging, it's important to distinguish between an error from the perspective of an individual operation and an error from the perspective of the overall program or system. Individual operations may well experience errors that are not error level log events for the overall program. You could say that an operation error is anything that prevents an operation from completing successfully, while a pro…

Log4j has the ability to filter log levels by subject matter for twenty years. In Java you end up having to use that a lot for this reason.

Logging in rust also does that, you can set logging levels for individual modules deep within your dependency tree.

Re: Log level 'error' should mean that something needs to be fixed

#117

Post by notatoad »

1718627440 wrote: post #103
notatoad wrote: post #63

Earlier quoted context omitted.

>How should the program know? if we're talking about logs from our own applications that we have written, the program should know because we can write it in a way that it knows. user-defined config should be verified before it is used. make a ping to port 25 to see if it works before you start using that config for actual operation. if it fails the verification step, that's not an error that needs to be logged.

So when the random error on a remote party happens at one time your system ignores it, bu when it happens at another time, it prevents the server from booting? That's a very brittle system.

log level error prevents your server from booting? i'm pretty sure that's not how logging works.

Re: Log level 'error' should mean that something needs to be fixed

#118

Post by esrauch »

Too wrote: post #92
layer8 wrote: post #76

> When implementing logging, it's important to distinguish between an error from the perspective of an individual operation and an error from the perspective of the overall program or system. Individual operations may well experience errors that are not error level log events for the overall program. You could say that an operation error is anything that prevents an operation from completing successfully, while a pro…

This is why it’s almost always wrong for library functions to log anything, even on ”errors”. Pass the status up through return values or exceptions. As a library author you have no clue as how an application might use it. Multi threading, retry loops and expected failures will turn what’s a significant event in one context into what’s not even worthy of a debug log in another. No rule without exceptions of course, o…

I think an example where libraries could sensibly log error is if you have a condition which is recoverable but may cause a significant slowdown, including a potential DoS issue, and the application owner can remediate.

You don't want to throw because destroying someone's production isn't worth it. You don't want to silent continue in that state because realistically there's no way for application owner to understand what is happening and why.

Re: Log level 'error' should mean that something needs to be fixed

#119

Post by notatoad »

tcpkump wrote: post #105
notatoad wrote: post #63

Earlier quoted context omitted.

>How should the program know? if we're talking about logs from our own applications that we have written, the program should know because we can write it in a way that it knows. user-defined config should be verified before it is used. make a ping to port 25 to see if it works before you start using that config for actual operation. if it fails the verification step, that's not an error that needs to be logged.

What about when the mail server endpoint has changed, and for whatever reason, this configuration wasn’t updated? This is a common scenario when dealing with legacy infrastructure in my experience.

the whole point of the essay here is that you should make a distinction between errors that you care about and plan to fix, and errors that you don't care about and don't intend to do anything about. and if you don't intend to do anything about it, it shouldn't be logged as error.

i'm following the author's example that an SMTP connection error is something you want to investigate and fix. if you have a different system with different assumptions where your response to a mailserver being unreachable is to ignore it, obviously that example doesn't apply for you. i'm not saying, and i don't think the author is saying that SMTP errors should always or never be logged as errors.

when the mailserver endpoint has changed, you should do the thing that makes sense in the context of your application. if it's not something that the person responsible for reviewing the logs needs to know about, don't log it. if it is, then log it.

Re: Log level 'error' should mean that something needs to be fixed

#120

Post by esrauch »

0x696C6961 wrote: post #88
layer8 wrote: post #76

> When implementing logging, it's important to distinguish between an error from the perspective of an individual operation and an error from the perspective of the overall program or system. Individual operations may well experience errors that are not error level log events for the overall program. You could say that an operation error is anything that prevents an operation from completing successfully, while a pro…

Libraries should not log, instead they should allow registering hooks which get called with errors and debug info.

It doesn't seem to work this way in practice, not least because most libraries will be transitive deps of the application owner.

I think creating the hooks is very close to just not doing anything here, if no one is going to use the hooks anyway then you might as well not have them.

Post reply on HN