What clicked for me is:
A span is a key-value attribute about some point in time event
A trace is a DAG of spans that tells you a story about some related events
What Are Traces and Spans in OpenTelemetry?
11–20 of 47 posts
Re: What Are Traces and Spans in OpenTelemetry?
#12While I do like the comprehensive writeup, there's something about the style which triggers my "it's AI-generated" reflex...
Re: What Are Traces and Spans in OpenTelemetry?
#13What clicked for me is: A span is a key-value attribute about some point in time event A trace is a DAG of spans that tells you a story about some related events
What do you mean exactly by "point in time event"?
As I understand it, a metric is information at a point in time.
A span however has a start timestamp and end timestamp, and is about a single operation that happens across that time.
https://opentelemetry.io/docs/specs/otel/metrics/
vs
Re: What Are Traces and Spans in OpenTelemetry?
#14The amount of additional code that it needs is horrible. We will now have to spend more brain juice on telemetry when working on a feature.
I work for Pydantic. We make Logfire, a commercial OTEL backend. But we’ve made wrappers around the OTEL SDKs in various languages that simplify configuration and usage. They can be used with any OTEL compatible backend (although we’d love if you try our SaaS offering):
- JavaScript / Typescript: https://github.com/pydantic/logfire-js
- Rust: https://github.com/pydantic/logfire-rust
- Python: https://github.com/pydantic/logfire
Re: What Are Traces and Spans in OpenTelemetry?
#15OTEL as a set of standards is admirable and ambitious, though in my experience actual implementation differs significantly between different vendors and they all seem to overcomplicate it.
Plus that tens of terabytes of data you have to store for a week worth of traces
That's why you sample just enough instead of storing everything
Re: What Are Traces and Spans in OpenTelemetry?
#16Earlier quoted context omitted.
Plus that tens of terabytes of data you have to store for a week worth of traces
That's why you sample just enough instead of storing everything
That sounds great until you have a massive issue that costs the company real money and leadership asks why you weren't logging everything in full fidelity?
We run with Debug logging on in prod for that reason too. We also ingest insane amounts of data but it does seem to be worth it for a sufficiently complex and important enough system to really have it all.
Re: What Are Traces and Spans in OpenTelemetry?
#17Re: What Are Traces and Spans in OpenTelemetry?
#18This is sort of all just a reframing of existing technologies. Span = an event (which is bascially just a log with an associated trace), and some data fields. Trace = a log for a request with a unique Id. A useful thing about opentelemetry is that there's auto-instrumentation so you can get this all out-of-the-box for most JVM apps. Of course you could probably log your queries instead, so it's not necessarily a game…
I always preach the isomorphism between traces and logs, but you left out the key thing. A span is a log entry associated with a trace, but the other key attributes of the span are its own unique identifier and a reference to the other event that caused the event. With those three attributes you can interpret the trace as a casual graph.
Re: What Are Traces and Spans in OpenTelemetry?
#19Earlier quoted context omitted.
Plus that tens of terabytes of data you have to store for a week worth of traces
That's why you sample just enough instead of storing everything
Sampling unconditionally at the start of the request is worth less than sampling at the end (so that your sample 1% of successful traces and be 100% of traces with issues).
Re: What Are Traces and Spans in OpenTelemetry?
#20This is sort of all just a reframing of existing technologies. Span = an event (which is bascially just a log with an associated trace), and some data fields. Trace = a log for a request with a unique Id. A useful thing about opentelemetry is that there's auto-instrumentation so you can get this all out-of-the-box for most JVM apps. Of course you could probably log your queries instead, so it's not necessarily a game…
yeah, but spans can have events!