Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Structured logging with slog

go.dev

Post reply on HN
Print view View original

81–90 of 174 posts

Re: Structured logging with slog

#81

Post by yencabulator »

mrweasel wrote: post #8

Admittedly I'm not a huge fan of having: slog.Info("hello, world", "user", os.Getenv("USER")) It's a little magical that "user" is a key. So what if you have multiple key-value pairs? Arguably it most likely going to be obvious which is the keys, but having every other value be a key and the rest values seems a little clumsy. I really like Pythons approach where you can have user="value" it makes things a bit more cl…

Personally I only intend to ever use LogAttrs/AddAttrs/GroupValue, for this reason. I really don't want to count evens and odds.

Re: Structured logging with slog

#82

Post by sa46 »

jupp0r wrote: post #43
mrweasel wrote: post #8

Admittedly I'm not a huge fan of having: slog.Info("hello, world", "user", os.Getenv("USER")) It's a little magical that "user" is a key. So what if you have multiple key-value pairs? Arguably it most likely going to be obvious which is the keys, but having every other value be a key and the rest values seems a little clumsy. I really like Pythons approach where you can have user="value" it makes things a bit more cl…

I think Uber chose a better approach for their Go logging library called Zap [1] logger.Info("failed to fetch URL", // Structured context as strongly typed Field values. zap.String("url", url), zap.Int("attempt", 3), zap.Duration("backoff", time.Second), ) They also have zap.Error(err) which generates the "error" key as a convention. [1] https://github.com/uber-go/zap

You can reproduce that example with slog verbatim by changing zap to slog.

For example: https://pkg.go.dev/golang.org/x/exp/slog#example-Group

Re: Structured logging with slog

#83

Post by corytheboyd »

scottlamb wrote: post #77
corytheboyd wrote: post #11

Structured logging is such an easy to gain, massive improvement to observability. Assuming you can pay for the log processor to make sense of it all :) I’ve been working on a side project to bring something like the DataDog log explorer to the local development environment. The prototype I made has already been extremely helpful in debugging issues in a very complex async ball of Rails code. Does something like that…

> Does something like that sound useful to other folks? Very much so! One piece I'd like to have is a good output format from my program. Right now I have stuff outputting logs in JSON format to stderr/local file (then picked up by the Opentelemetry OTLP collector, Datadog agent, AWS CloudWatch, whatever) and the traces/spans actually sent from my process to the collector over the network. It baffles me why the trace…

I hadn't even considered collecting traces/spans in this way yet, and have taken the approach of "stuff outputting logs in JSON format to stderr/local file". I usually end up writing a (temporary, structured) log message with the relevant span tags, but wouldn't it be much better to run the actual trace/span code and be able to verify it locally without the ad-hoc log message?

The prototype I built is a web application that creates websocket connections, and if those connections receive messages that are JSON, log lines are added. Columns are built dynamically as log messages arrive, and then you can pick which columns to render in the table. If you're curious here's the code, including a screenshot: https://github.com/corytheboyd-smartsheet/json-log-explorer

With websockets, it's very easy to use websocketd (http://websocketd.com), which will watch input files for new lines, and write them verbatim as websocket messages to listeners (the web app).

To make the idea real, would want to figure out how to not require the user to run websocketd out of band, but watching good ol' files is dead simple, and very easy to add to most code (add a new log sink, use existing log file, etc.)

Re: Structured logging with slog

#84

Post by yencabulator »

mlhpdx wrote: post #76
ben0x539 wrote: post #51

Earlier quoted context omitted.

I think go loggers have tried to move away from passing log entries as maps for performance reasons.

That seems like a problem that should be solved. Logging structured data is a very basic expectation.

Avoiding the map allocation & construction cost is way harder than avoiding the use of a map, like zap and slog.LogAttrs do.

Re: Structured logging with slog

#85

Post by Groxx »

bpicolo wrote: post #48
jupp0r wrote: post #43

Earlier quoted context omitted.

I think Uber chose a better approach for their Go logging library called Zap [1] logger.Info("failed to fetch URL", // Structured context as strongly typed Field values. zap.String("url", url), zap.Int("attempt", 3), zap.Duration("backoff", time.Second), ) They also have zap.Error(err) which generates the "error" key as a convention. [1] https://github.com/uber-go/zap

As mentioned in the article, Zap's SugaredLogger has roughly the same thing https://pkg.go.dev/go.uber.org/zap sugar.Infow("failed to fetch URL", "url", "http://example.com", "attempt", 3, "backoff", time.Second, ) Really slog presents mostly the same two APIs as Zap, with a single entrypoint.

`[level]w` supports both key/value pairs and individual `zap.Type(key, value)` values combined fwiw, so for that part the API is essentially the same too

Re: Structured logging with slog

#86

Post by llimllib »

tony_cannistra wrote: post #46
kusha wrote: post #31

Oof. We just converted all of our logging to zap[0] to get structured JSON logging for downstream parsing. Wonder how the perf stacks up. [0]: https://github.com/uber-go/zap

It looks like they've included slog in their performance benchmarks, which show zap as considerably more performant (though I don't really understand the benchmark).

http://bench.zerolog.io/ Is a useful set of benchmarks

Re: Structured logging with slog

#87

Post by codetrotter »

richardwhiuk wrote: post #62
mrweasel wrote: post #8

Admittedly I'm not a huge fan of having: slog.Info("hello, world", "user", os.Getenv("USER")) It's a little magical that "user" is a key. So what if you have multiple key-value pairs? Arguably it most likely going to be obvious which is the keys, but having every other value be a key and the rest values seems a little clumsy. I really like Pythons approach where you can have user="value" it makes things a bit more cl…

Rust's slog ( https://docs.rs/slog/ ) does: use slog::info; ... info!("hello, world"; "user" => std::env::var("USER"));

Yup! Love the Rust slog crate. Initially thought OP was gonna be about Rust slog, before I saw the domain mention Go.

Re: Structured logging with slog

#88

Post by jasonhansel »

I must admit: I'm not a huge fan of structured logging, beyond simple use cases like tagging messages by the thread that produced them. If you want something machine-readable, use a dedicated metrics system, analytics database, or document store. If you want something human-readable, structured logging will only make things worse.

Re: Structured logging with slog

#89

Post by jimmcslim »

jasonhansel wrote: post #88

I must admit: I'm not a huge fan of structured logging, beyond simple use cases like tagging messages by the thread that produced them. If you want something machine-readable, use a dedicated metrics system, analytics database, or document store. If you want something human-readable, structured logging will only make things worse.

I feel what is missing here is message templates [1] - the logging API should permit the key-value pairs to be substituted into a template which results in a human-readable message, while preserving the KV data separately. Take a hash of the template and add it as a KV pair so that messages of the same type can be easily filtered.

[1] https://messagetemplates.org

Re: Structured logging with slog

#90

Post by andreygrehov »

jasonhansel wrote: post #88

I must admit: I'm not a huge fan of structured logging, beyond simple use cases like tagging messages by the thread that produced them. If you want something machine-readable, use a dedicated metrics system, analytics database, or document store. If you want something human-readable, structured logging will only make things worse.

Structured logging is not meant for humans to read. It's meant for machines to read and represent in a human readable format. Additionally, these logs can _later_ be streamed into a metrics system, analytics database, or a document store. Sort of in a plug & play fashion.
Post reply on HN