Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Log level 'error' should mean that something needs to be fixed

utcc.utoronto.ca

Post reply on HN
Print view View original

221–230 of 313 posts

Re: Log level 'error' should mean that something needs to be fixed

#221

Post by dolmen »

ivan_gammel wrote: post #104
1718627440 wrote: post #101

Earlier quoted context omitted.

Why? Whats wrong with logging it and passing the log object to the caller? The caller can still modify the log entry however it pleases?

Practicality. It is excessive for client code to calibrate library logging level. It’s ok to do it in logging configuration, but having an entry for every library there is also excessive. It is reasonable to expect that dev/staging may have base level at DEBUG and production will have base level at INFO, so that a library following the convention will not require extra effort to prevent log spam in production. Yes, w…

TLDR: I agree.

A library might also be used in multiple place, maybe deeply in a dependency stack, so the execution context (high level stack) matters more than which library got a failure.

So handling failures should stay in the hands of the developer calling the library and this should be a major constraint for API design.

Re: Log level 'error' should mean that something needs to be fixed

#222

Post by dolmen »

kelnos wrote: post #126
0x696C6961 wrote: post #88

Earlier quoted context omitted.

Libraries should not log, instead they should allow registering hooks which get called with errors and debug info.

I think this is useful for libraries in a language like C, where there is no standardized logging framework, so there's no way for the application to control what the library logs. But in a language (Java, Rust, etc.) where there are standard, widely-used logging frameworks that give people fine-grained control over what gets logged, libraries should just use those frameworks. (Even in C, though... errors should be s…

Log4J style logging is effectively a hook system. But it is too easy to badly use it with too high level and delegate level fixing to the end user.

Re: Log level 'error' should mean that something needs to be fixed

#223

Post by energy123 »

Kwpolska wrote: post #162
Too wrote: post #153

Earlier quoted context omitted.

Simple: include those relevant details in the exceptions instead of hiding them.

Not all problems cause exceptions.

That's a matter of good taste, but there's nothing preventing you from throwing exceptions on every issue and requiring consumers to handle them

Re: Log level 'error' should mean that something needs to be fixed

#224

Post by demurgos »

Lvl999Noob wrote: post #219
usefulcat wrote: post #187

Earlier quoted context omitted.

Ideally a logging library should at least not make it easy to make that kind of mistake.

This isn't really something the logging library can do. If the language provides a string interpolation mechanism then that mechanism is what the programmers will reach for first. And the library cannot know that interpolation happened because the language creates the final string before passing it in. If you want the builtin interpolation to become a noop in the face runtime log disabling then the logging library ha…

I feel like there's a parallel with SQL where you want to discourage manual interpolation. Taking inspiration from it may help: you may not fully solve it but there are some API ideas and patterns.

A logging framework may have the equivalent of prepared statements. You may also nudge usage where the raw string API is `log.traceRaw(String rawMessage)` while the parametrized one has the nicer naming `log.trace(Template t, param1, param2)`.

Re: Log level 'error' should mean that something needs to be fixed

#225

Post by rr808 »

cogman10 wrote: post #132
Too wrote: post #92

Earlier quoted context omitted.

This is why it’s almost always wrong for library functions to log anything, even on ”errors”. Pass the status up through return values or exceptions. As a library author you have no clue as how an application might use it. Multi threading, retry loops and expected failures will turn what’s a significant event in one context into what’s not even worthy of a debug log in another. No rule without exceptions of course, o…

Depending on the language and logging framework, debug/trace logging can be acceptable in a library. But you have to be extra careful to make sure that it's ultimately a no-op. A common problem in Java is someone will drop a log that looks something like this `log.trace("Doing " + foo + " to " + bar);` The problem is, especially in a hot loop, that throw away string concatenation can ultimately be a performance probl…

Still quite like the windows log approach which (if logged) stores the template as just the id, with the values, saving lots of storage as well eg 123, foo, bar. You can concatenate in the reader.

Re: Log level 'error' should mean that something needs to be fixed

#226

Post by Retric »

dpark wrote: post #169
cwillu wrote: post #159

Earlier quoted context omitted.

Those details don't belong in the error log level, that's what info or trace is for.

They were replying to a person who says “it’s almost always wrong for library functions to log anything”. Not just errors.

If it’s not your code how is a log useful vs returning an error?

Even relatively complex operations like say convert this document into a PDF etc basically only has two useful states either it worked or something specific failed at which point just tell me that thing.

Now independent software like web servers or database can have useful logs because they have completely independent interfaces with the outside world. But I call libraries they don’t call me.

Re: Log level 'error' should mean that something needs to be fixed

#227

Post by dpark »

TZubiri wrote: post #215
dpark wrote: post #170

Earlier quoted context omitted.

Assuming everything is idempotent is a tall order. There are a lot of libraries that haven non-idempotent actions. There are a lot of inputs that can be problematic to log, too.

Say like opening a file? I guess in those cases standard practice is for lib to return a detailed error yeah. As far as traces, trying to solve issues that depend on external systems is indeed a tall order for your code. Isn't it beyond the scope of the thing being programmed.

I don’t really understand what you mean about opening files. Is this just an example of an idempotent action or is there some specific significance here?

Either way logging the input (file name) is notably not sufficient for debugging if the file can change between invocations. The action can be idempotent and still be affected by other changes in the system.

> trying to solve issues that depend on external systems is indeed a tall order for your code. Isn't it beyond the scope of the thing being programmed.

If my program is broken I need it fixed regardless of why it’s broken. The specific example here of a file changing is likely to manifest as flakiness that’s impossible to diagnose without detailed logs from within the library.

Re: Log level 'error' should mean that something needs to be fixed

#228

Post by dolmen »

lanstin wrote: post #211
29athrowaway wrote: post #123

Input errors do not need fixing, so no.

If they cause your customers to ditch your product but calling them and saying "your calls are all getting 4xx because you are not putting the state code into the call parameters" would keep them as customers, then you would be wise to make that communication.

But first ensure that the input error is properly reported to the client in the response body (ideally in a structured way), so the client could have figured out by himself.

If a fix is needed on your side for this matter, having a conversation with a customer might be useful before breaking more stuff. ("We have no state code in EU. Why is that mandatory?").

Re: Log level 'error' should mean that something needs to be fixed

#229

Post by Insanity »

Coincidentally was reviewing code yesterday that had a confusing/contradictory statement..

  error_msg = "xyz went wrong"
  log.warn(error_msg)
My comment on the CR was about this being an inherent contradiction and incredibly confusing to know if it's actually an error or a warning..

Re: Log level 'error' should mean that something needs to be fixed

#230

Post by TZubiri »

dpark wrote: post #227
TZubiri wrote: post #215

Earlier quoted context omitted.

Say like opening a file? I guess in those cases standard practice is for lib to return a detailed error yeah. As far as traces, trying to solve issues that depend on external systems is indeed a tall order for your code. Isn't it beyond the scope of the thing being programmed.

I don’t really understand what you mean about opening files. Is this just an example of an idempotent action or is there some specific significance here? Either way logging the input (file name) is notably not sufficient for debugging if the file can change between invocations. The action can be idempotent and still be affected by other changes in the system. > trying to solve issues that depend on external systems i…

I was just trying to think of an example of a non idempotent function. As in it depends on an external IO device.

I will say that error handling and logging in general is one of my weakpoints, but I made a comment about my approach so far being dbg/pdb based, attaching a debugger and creating breakpoints and prints ad-hoc rather than writing them in code. I'm sure there's reasons why it isn't used as much and logging in code is so much more common, but I have faith that it's a path worth specializing in.

Back to the file reading example, for a non-idempotent function. Considering we are using an encapsulating approach we have to split ourselves into 3 roles. We can be the IO library writer, we can be the calling code writer, and we can be an admin responsible for the whole product. I think a common trap engineers fall for is trying to keep all of the "global" context (or as much as they can handle) at all times.

In this case of course we wouldn't be writing the non-idempotent library, so of course that's not a hat we wear, do not quite care about the innards of the function and its state, rather we have a well defined set of errors that are part of the interface of the function (EINVAL, EACCES, EEXIST).

In this sense we respect the encapsulation boundaries and are provided the information necessary by the library. If we ever need to dive into the actual library code, first the encapsulation is broken and we are dealing with a leaky abstraction, second we just dive into the library code, (or the filesystem admin logs themselves).

It's not precisely the type of responsibility that can be handled at design time and in code anyways, when we code we are wearing the calling-module programmer hat. We cannot think of everything that the sysadmin might need at the time of experiencing an error, we have to think that they will be sufficiently armed with enough tools to gather the information necessary with other tools. And thank god for that! checking /proc/fs and looking at crash dumps, and attaching processes with dbg will yield far better info than relying on whatever print statements you somehow added to your program.

Anyways at least that's my take on the specific example of glibc-like implementations of POSIX file operations like open(). I'm sure the implications may change for other non-idempotent functions, but at some point, talking about specifics is a bit more productive than talking in the abstract.

Post reply on HN