Live data from Hacker News

Log level 'error' should mean that something needs to be fixed

utcc.utoronto.ca

61–70 of 313 posts

Re: Log level 'error' should mean that something needs to be fixed

#61

Earlier quoted context omitted.

What if you are integrated to a third party app and it gives you 5xx once? What do you log it as, and let's say after a retry it is fine.

I would log a warning when an attempt fails, and an error when the final attempt fails.

You are not the OP, but I think I was trying to point out this example case in relation to their descriptions of Error/Warnings.

This scenario may or may not yield in data/state loss, it may also be something that you, yourself can't immediately fix. And if it's temporary, what is the point of creating an issue and prioritizing.

I guess my point is that to any such categorization of errors or warnings there are way too many counter examples to be able to describe them like that.

So I'd usually think that Errors are something that I would heuristically want to quickly react to and investigate (e.g. being paged, while Warnings are something I would periodically check in (e.g. weekly).

Re: Log level 'error' should mean that something needs to be fixed

#63

“cannot contact port 25 on ” may very well be a configuration error. How should the program know?

>How should the program know?

if we're talking about logs from our own applications that we have written, the program should know because we can write it in a way that it knows.

user-defined config should be verified before it is used. make a ping to port 25 to see if it works before you start using that config for actual operation. if it fails the verification step, that's not an error that needs to be logged.

Re: Log level 'error' should mean that something needs to be fixed

#64
post #45

How I'd personally like to treat them: - Critical / Fatal: Unrecoverable without human intervention, someone needs to get out of bed, now. - Error : Recoverable without human intervention, but not without data / state loss. Must be fixed asap. An assumption didn't hold. - Warning: Recoverable without intervention. Must have an issue created and prioritised. ( If business as usual, this could be downgrading to INFO. )…

What if you are integrated to a third party app and it gives you 5xx once? What do you log it as, and let's say after a retry it is fine.

As always „it depends”

- info - when this was expected and system/process is prepared for that (like automatic retry, fallback to local copy, offline mode, event driven with persistent queue etc) - warning - when system/process was able to continue but in degraded manner, maybe leaving decision to retry to user or other part of system, or maybe just relying on someone checking logs for unexpected events, this of course depends if that external system is required for some action or in some way optional - error - when system/process is not able to continue and particular action has been stopped immediately, this includes situation where retry mechanism is not implemented for step required for completion of particular action - fatal - you need to restart something, either manually or by external watchdog, you don’t expect this kind of logs for simple 5xx

Re: Log level 'error' should mean that something needs to be fixed

#66
post #45

How I'd personally like to treat them: - Critical / Fatal: Unrecoverable without human intervention, someone needs to get out of bed, now. - Error : Recoverable without human intervention, but not without data / state loss. Must be fixed asap. An assumption didn't hold. - Warning: Recoverable without intervention. Must have an issue created and prioritised. ( If business as usual, this could be downgrading to INFO. )…

What if you are integrated to a third party app and it gives you 5xx once? What do you log it as, and let's say after a retry it is fine.

Well, the GPs criteria are quite good. But what you should actually do depends on a lot more things than the ones you wrote in your comment. It could be so irrelevant to only deserve a trace log, or so important to get a warning.

Also, you should have event logs you can look to make administrative decisions. That information surely fits into those, you will want to know about it when deciding to switch to another provider or renegotiate something.

Re: Log level 'error' should mean that something needs to be fixed

#67
post #59

Earlier quoted context omitted.

What if you are integrated to a third party app and it gives you 5xx once? What do you log it as, and let's say after a retry it is fine.

This might be controversial, but I'd say if it's fine after a retry, then it doesn't need a warning. Because what I'd want to know is how often does it fail , which is a metric not a log. So expose as a metric not a log. If feeding logs into datadog or similar is the only way you're collecting metrics, then you aren't treating your observablity with the respect it deserves. Put in real counters so you're not just rea…

> If the third party being down has a knock-on effect to your own system functionality / uptime, then it needs to be a warning or error, but you should also put in the backlog a ticket to de-couple your uptime from that third-party, be it retries, queues, or other mitigations ( alternate providers? ).

How do you define uptime? What if e.g. it's a social login / data linking and that provider is down? You could have multiple logins and your own e-mail and password, but you still might lose users because the provider is down. How do you log that? Or do you only put it as a metric?

You can't always easily replace providers.

Re: Log level 'error' should mean that something needs to be fixed

#68

Earlier quoted context omitted.

I would log a warning when an attempt fails, and an error when the final attempt fails.

You are not the OP, but I think I was trying to point out this example case in relation to their descriptions of Error/Warnings. This scenario may or may not yield in data/state loss, it may also be something that you, yourself can't immediately fix. And if it's temporary, what is the point of creating an issue and prioritizing. I guess my point is that to any such categorization of errors or warnings there are way t…

Like so many things in this industry the point is establishing a shared meaning for all the humans involved, regardless of how uninvolved people think.

That being said, I find tying the level to expected action a more useful way to classify them.

Re: Log level 'error' should mean that something needs to be fixed

#70
post #59

Earlier quoted context omitted.

What if you are integrated to a third party app and it gives you 5xx once? What do you log it as, and let's say after a retry it is fine.

This might be controversial, but I'd say if it's fine after a retry, then it doesn't need a warning. Because what I'd want to know is how often does it fail , which is a metric not a log. So expose as a metric not a log. If feeding logs into datadog or similar is the only way you're collecting metrics, then you aren't treating your observablity with the respect it deserves. Put in real counters so you're not just rea…

> This might be controversial, but I'd say if it's fine after a retry, then it doesn't need a warning. > > Because what I'd want to know is how often does it fail, which is a metric not a log.

It’s not controversial; you just want something different. I want the opposite: I want to know why/how it fails; counting how often it does is secondary. I want a log that says "I sent this payload to this API and I got this error in return", so that later I can debug if my payload was problematic, and/or show it to the third party if they need it.

Post reply on HN