OTel isn’t going well • Hacker News Board

Re: OTel isn’t going well

#31

Post by MathMonkeyMan » Sat, Aug 22, 2026, 6:11 AM UTC

EdSchouten wrote: post #9

What always puzzles me about OpenTelemetry is that tracing, metrics and logs are all designed independently. I wish there was a way I could just annotate my code base once, and let the ultimate decision to expose something as a metric/log/trace be dynamic at runtime. For example, if I look at a graph in monitoring dashboard and see something suspicious, I’d like to say: “The next time something like this occurs again…

Tracing is the most general of them, and the most expensive unless you're careful with the implementation.

Trace spans are time-delimited units of "stuff that happened", with a tree relationship among the spans, and each span can have arbitrary tags (key/value pairs) and events (time/value).

From that, if you chose, you could derive metrics and logs. The trick is to start with tracing and to actually put it in your program, rather than trying to mostly-automatically tack it on later.

Re: OTel isn’t going well

#32

Post by osener » Sat, Aug 22, 2026, 6:18 AM UTC

I like the end result of OpenTelemetry tracing when using Axiom and the like, but the SDKs have been a nightmare. Too much emphasis on automatic instrumentation, Java-isms, everything is stateful and abstracted away.

It can do distributed tracing of otherwise traditional long running microservices, but breaks down when your functions are distributed like in durable execution engines, Cloudflare Workflows, “functions” that span hours/days/weeks and steps that retry many times.

I had to reverse engineer how SDKs work and how tracing UIs display data so I could make simpler functions that fit wider variety of runtimes and more freely parent spans, start spans and end them from different function instances.

I think most of the API and terminology complexity is self inflicted. Would love to see a rebooted developer experience that is less Kubernates-brained.

Re: OTel isn’t going well

#33

Post by jauntywundrkind » Sat, Aug 22, 2026, 6:26 AM UTC

masterj wrote: post #21

cyberax wrote: post #13

I disagree. I'm an observability geek, and OTel is... fine. It's missing a few things that I'd like, but I was able to implement them myself. I guess the major design issue is that the sampling decision is made at the _start_ of the segment. So I hacked up a few improvements: 1. Ability to mark segments as "boring", so they are dropped before the export. For things like healthchecks, empty "get the pending jobs" quer…

HN always grumbles about OTel, but I agree. It's fine, and important: https://jeremymorrell.dev/blog/opentelemetry-and-the-value-o... I understand the author's perspective in the linked article, but none of that data shows a project in trouble? Some languages have more resources than others, but those all look like healthy open source projects

I'd make a wager that things would go better smoother faster if folks tried more stuff, ventures forth more on their own. It's obviously not great that there's no semantic convention that's perfect and just works for everything, and yeah it takes a while. I feel like the real data I'd want is who else, how many people show up to say they've tried something. Is that happening? Whether specs are really good enough advance or not, to me, is often whether enough people have tried it to find out.

The net of this is, otel is a very flexible system you can use and adapt in all kinds of ways and while the spec is important, using the toolkit to FAFO yourself, ahead of any beaten path, should really be encouraged. That's the message I'd want to see being radiated out about otel.

Re: OTel isn’t going well

#34

Post by fuzzy2 » Sat, Aug 22, 2026, 7:09 AM UTC

EdSchouten wrote: post #9

What always puzzles me about OpenTelemetry is that tracing, metrics and logs are all designed independently. I wish there was a way I could just annotate my code base once, and let the ultimate decision to expose something as a metric/log/trace be dynamic at runtime. For example, if I look at a graph in monitoring dashboard and see something suspicious, I’d like to say: “The next time something like this occurs again…

I just don’t get this sentiment. How would you represent metrics as traces? You cannot. Even reconstructing traces from logs would be challenging at best. How would you get, say, Garbage Collector metrics from logs or traces? You cannot.

There is no magic bullet. Observability isn’t something you can just slap on and call it a day. While traces and logs might share superficial similarities, they are not the same. And metrics are something else altogether. Trying to somehow unify them would be a prime example of "wrong abstraction".

> “The next time something like this occurs again, please save me a trace.”

The building blocks for this exist. The observability platform must simply (haha) implement the pattern detectors and use them for sampling decisions.

Re: OTel isn’t going well

#35

Post by gertburger » Sat, Aug 22, 2026, 7:27 AM UTC

I've found their django instrumentation to be kinda useless for larger apps.

The only choices you get is full auto instrumentation, which breaks most non-trivial apps, or zero assistance/documentation.

There is no in-between where I can inject the functionality required in a way that is compatible with the application.

Re: OTel isn’t going well

#36

Post by anygivnthursday » Sat, Aug 22, 2026, 7:27 AM UTC

fuzzy2 wrote: post #34

EdSchouten wrote: post #9

What always puzzles me about OpenTelemetry is that tracing, metrics and logs are all designed independently. I wish there was a way I could just annotate my code base once, and let the ultimate decision to expose something as a metric/log/trace be dynamic at runtime. For example, if I look at a graph in monitoring dashboard and see something suspicious, I’d like to say: “The next time something like this occurs again…

I just don’t get this sentiment. How would you represent metrics as traces? You cannot. Even reconstructing traces from logs would be challenging at best. How would you get, say, Garbage Collector metrics from logs or traces? You cannot. There is no magic bullet. Observability isn’t something you can just slap on and call it a day. While traces and logs might share superficial similarities, they are not the same. And…

I am not sure if this is what they mean, but e.g. with Micrometer in Java you can instrument your code once with observations that produces observation events, then you can register handlers that can turn them into metrics, or logs, or traces without having to instrument your code three times.

https://docs.micrometer.io/micrometer/reference/observation....

Re: OTel isn’t going well

#37

Post by [deleted] » Sat, Aug 22, 2026, 7:28 AM UTC

arcanemachiner wrote: post #22

cyberax wrote: post #14

Earlier quoted context omitted.

OTEL metrics are a bit awkward, but they work just fine with Prometheus. Jaeger uses the OTLP protocol nowadays. So it _is_ OTEL.

What, so people don't like OTel, but they like Jaeger, which implements an OTel spec? (I'm a noob to this subject, if that wasn't obvious.)

[deleted]

Re: OTel isn’t going well

#38

Post by rm » Sat, Aug 22, 2026, 7:43 AM UTC

gertburger wrote: post #35

I've found their django instrumentation to be kinda useless for larger apps. The only choices you get is full auto instrumentation, which breaks most non-trivial apps, or zero assistance/documentation. There is no in-between where I can inject the functionality required in a way that is compatible with the application.

Could you please elaborate a bit on what is not working for you?

Re: OTel isn’t going well

#39

Post by tete » Sat, Aug 22, 2026, 7:55 AM UTC

OpenTelemtry is the perfect example of an overengineered mess.

While I usually think that at least having some standard that people agree on I think OpenTelemtry should be dropped.

A lot of the less popular alternatives (just going with Prometheus, Victoriametrics, etc) are de-facto competing smaller standards and a lot better both in terms of less added complexity and the results you get.

I think OpenTelemetry turned metrics into a farce. In many situations even self-rolled telemetry works better even with the added stuff. The annoying thing is that OpenTelemtry is that big standard now one kind of has to to add compatibility. So please, if you write software, make sure you don't lock yourself into OTel.

Re: OTel isn’t going well

#40

Post by Flamkuchlo » Sat, Aug 22, 2026, 7:57 AM UTC

anygivnthursday wrote: post #36

fuzzy2 wrote: post #34

Earlier quoted context omitted.

I just don’t get this sentiment. How would you represent metrics as traces? You cannot. Even reconstructing traces from logs would be challenging at best. How would you get, say, Garbage Collector metrics from logs or traces? You cannot. There is no magic bullet. Observability isn’t something you can just slap on and call it a day. While traces and logs might share superficial similarities, they are not the same. And…

I am not sure if this is what they mean, but e.g. with Micrometer in Java you can instrument your code once with observations that produces observation events, then you can register handlers that can turn them into metrics, or logs, or traces without having to instrument your code three times. https://docs.micrometer.io/micrometer/reference/observation....

The problem is not the instrumentation but the way everyone of them work.

A metric is a point in time. A metric is very small but you have a lot of them.

A log is when something is happening but you need to log it out. A logline is heavy and has a lot of context. User id, message, etc.

A trace needs to start at the request level and tracing until the response. This is the slowest and heaviest operation.

How do you decide when to suddenly do the trace and send it? IF you always do the trace, you have to pay for the overhead of that tracing constantly.