61–70 of 123 posts
Earlier quoted context omitted.
OTEL metrics are a bit awkward, but they work just fine with Prometheus. Jaeger uses the OTLP protocol nowadays. So it _is_ OTEL.
What, so people don't like OTel, but they like Jaeger, which implements an OTel spec? (I'm a noob to this subject, if that wasn't obvious.)
Earlier quoted context omitted.
I am not sure if this is what they mean, but e.g. with Micrometer in Java you can instrument your code once with observations that produces observation events, then you can register handlers that can turn them into metrics, or logs, or traces without having to instrument your code three times. https://docs.micrometer.io/micrometer/reference/observation....
The problem is not the instrumentation but the way everyone of them work. A metric is a point in time. A metric is very small but you have a lot of them. A log is when something is happening but you need to log it out. A logline is heavy and has a lot of context. User id, message, etc. A trace needs to start at the request level and tracing until the response. This is the slowest and heaviest operation. How do you de…
Search keyword: "Adaptive sampling"
I find the entire observability space to quite a poor experience, at least in the self-hosted space. Tried both grafana route and signoz and neither seems particularly pleasant
Earlier quoted context omitted.
The problem is not the instrumentation but the way everyone of them work. A metric is a point in time. A metric is very small but you have a lot of them. A log is when something is happening but you need to log it out. A logline is heavy and has a lot of context. User id, message, etc. A trace needs to start at the request level and tracing until the response. This is the slowest and heaviest operation. How do you de…
Logs and metrics are both derived from events. A log takes the whole event and records it somewhere. A metric takes some numeric value from the event, aggregates it over time, and records it periodically. You can reconstruct a metric from logs for the underlying events. A trace is a period of execution between two events. You could record a trace as a pair of log entries, or one log entry at the end. You can then rec…
You don't have a metric 'person logged in' because you would need to scrape the metric at the moment a person logged in.
You have a metric called 'overall people have logged in so far' and you do math on it.
The 'person logged in' is an event you log out.
Earlier quoted context omitted.
The problem is not the instrumentation but the way everyone of them work. A metric is a point in time. A metric is very small but you have a lot of them. A log is when something is happening but you need to log it out. A logline is heavy and has a lot of context. User id, message, etc. A trace needs to start at the request level and tracing until the response. This is the slowest and heaviest operation. How do you de…
What? All of this has been solved for a long time. How do you think hyperscalers do this? Search keyword: "Adaptive sampling"
Tracing traces a particular event.
I'm quite aware of the difference between sampling, tracing and profiling.
Earlier quoted context omitted.
The problem is not the instrumentation but the way everyone of them work. A metric is a point in time. A metric is very small but you have a lot of them. A log is when something is happening but you need to log it out. A logline is heavy and has a lot of context. User id, message, etc. A trace needs to start at the request level and tracing until the response. This is the slowest and heaviest operation. How do you de…
At that point you almost might as well just log everything. The decision logic is likely about as complex as just doing it. Then I suppose you have a watchdog task that fires off every, say, 15 minutes or an hour or something, looks at the collected data, and either decides to keep it or trash it while recording a tiny "nothing interesting" datapoint.
All the log ingestion systems i have seen were bigger elastic search clusters.