Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The problem with OpenTelemetry

cra.mr

Post reply on HN
Print view View original

71–80 of 178 posts

Re: The problem with OpenTelemetry

#71

Post by nimih »

tnolet wrote: post #50
hahn-kev wrote: post #42

Earlier quoted context omitted.

Yeah but who wants to contribute to an SDK for a service that you need to pay for? That would be like if Oracle DB was open to contribution

Sentry is self hostable. https://develop.sentry.dev/self-hosted/

Sentry is technically self-hostable, but they provide no deployment guidance beyond running the giant blob of services/microservices (including instances of postgres, redis, memcache, clickhouse, and kafka) as a single docker-compose thing. I get why they do this and think it's totally reasonable of them, but Sentry is a very complicated piece of software and takes substantially more work IME to both get up and running and maintain compared to other open-source self-hosted observability/monitoring/telemetry software I've had the pleasure of working with.

Re: The problem with OpenTelemetry

#72

Post by miohtama »

nimih wrote: post #71
tnolet wrote: post #50

Earlier quoted context omitted.

Sentry is self hostable. https://develop.sentry.dev/self-hosted/

Sentry is technically self-hostable, but they provide no deployment guidance beyond running the giant blob of services/microservices (including instances of postgres, redis, memcache, clickhouse, and kafka) as a single docker-compose thing. I get why they do this and think it's totally reasonable of them, but Sentry is a very complicated piece of software and takes substantially more work IME to both get up and runni…

Our Linux devops engineer, who had not used Sentry before, set up a self-hosted Sentry in a day.

Re: The problem with OpenTelemetry

#73

Post by zeeg »

serverlessmom wrote: post #61
zeeg wrote: post #57

Earlier quoted context omitted.

Author here. OpenTelemetry is not competitive to us (it doesn’t do what we do in plurality), and we specifically want to see the open tracing goals succeed. I was pretty clear about that in the post though.

I think that it's disingenuous to say OpenTelemetry and Sentry aren't in competition. I think it would be good news for Sentry if DT were split from the project, and instrumentation and performance monitoring weren't commoditized by broad adoption of those parts of the OpenTelemetry project. I think you, the author, stand to benefit directly from a breakup of OpenTelemetry, and a refusal to acknowledge your own bias…

We just rewrote our most heavily used SDK to run on top of OTel. What do we gain from it failing?

We also make most of our revenue from errors which don’t have an open protocol implementation outside of our own.

Re: The problem with OpenTelemetry

#74

Post by codereflection »

I understand what the author is saying, but vendor lock-in with closed-source observability platforms is a significant challenge, especially for large organizations. When you instrument hundreds or thousands of applications with a specific tool, like the Datadog Agent, disentangling from that tool becomes nearly impossible without a massive investment of engineering time. In the Platform Engineering professional services space, we see this problem frequently. Enterprises are growing tired of big observability platform lock-in, especially when it comes to Datadog's opaque nature of your spend on their products, for example.

One of the promises of OTEL is that it allows organizations to replace vendor-specific agents with OTEL collectors, allowing the flexibility of the end observability platform. When used with an observability pipeline (such as EdgeDelta or Cribl), you can re-process collected telemetry data and send it to another platform, like Splunk, if needed. Consequently, switching from one observability platform to another becomes a bit less of a headache. Ironically, even Splunk recognizes this and has put substantial support behind the OTEL standard.

OTEL is far from perfect, and maybe some of these goals are a bit lofty, but I can say that many large organizations are adopting OTEL for these reasons.

Re: The problem with OpenTelemetry

#75

Post by tapoxi »

miohtama wrote: post #72
nimih wrote: post #71

Earlier quoted context omitted.

Sentry is technically self-hostable, but they provide no deployment guidance beyond running the giant blob of services/microservices (including instances of postgres, redis, memcache, clickhouse, and kafka) as a single docker-compose thing. I get why they do this and think it's totally reasonable of them, but Sentry is a very complicated piece of software and takes substantially more work IME to both get up and runni…

Our Linux devops engineer, who had not used Sentry before, set up a self-hosted Sentry in a day.

Yeah, it works for a time, but they don't support on-premise versions and they don't offer a Helm chart install, its all community based.

I tried it for well over a year, and there are so many moving parts and so many "best guesses" from the community that we had to rip it out. There's a lot of components, sentry, sentry-relay, snuba, celery, redis, clickhouse, zookeeper (for clickhouse), kafka, zookeeper (for kafka), maybe even elasticsearch for good measure. It did work for a time, but there are so many moving parts that required care and feeding it would inevitably break down at some point.

Problem is I can't ship data to their SaaS version because we have PHI and our contracts forbid it, even if scrubbed, so I had to settle on OTEL.

Re: The problem with OpenTelemetry

#76

Post by zellyn »

Are they basically just saying that the OpenTelemetry client APIs should be split from the rest of the pieces of the project, and versioned super conservatively?

The simple API they describe is basically there in OTel. The API is larger, because it also does quite a few other things (personally, I think (W3C) Baggage is important too), but as a library author I should need only the client APIs to write to.

When implementing, you're free to plug in Providers that use OpenAPI-provided plumbing, but you can equally well plug in Providers from DataDog or Sentry or whatever.

Unless I'm missing something, any further complaints could be solved by making sure the Client APIs (almost) never have backward-incompatible changes, and are versioned separately.

Re: The problem with OpenTelemetry

#77

Post by politelemon »

marcosdumay wrote: post #66
doctorpangloss wrote: post #16

I don’t know what the Sentry guy is really saying - I mean you can write whatever code you want, go for it man. But I do have to “pip uninstall sentry-sdk” in my Dockerfile because it clashes with something I didn’t author. And anyway, because it is completely open source, the flaws in OpenTelemetry for my particular use case took an hour to surmount, and vitally, I didn’t have to pay the brain damage cost most devel…

> eventually everyone is going to learn how to use Kubernetes That seems obviously true... yet, there are so many people out there that seem unable to learn it that I don't think it's a reliable prediction.

> unable

I wouldn't equate unwillingness or not needing it to inability to learn

Re: The problem with OpenTelemetry

#78

Post by zeeg »

codereflection wrote: post #74

I understand what the author is saying, but vendor lock-in with closed-source observability platforms is a significant challenge, especially for large organizations. When you instrument hundreds or thousands of applications with a specific tool, like the Datadog Agent, disentangling from that tool becomes nearly impossible without a massive investment of engineering time. In the Platform Engineering professional serv…

I totally agree I just wish we could do it in a way that doesn’t try to lump every problem into the same bucket. I don’t see what it achieves personally, and I think it’s limiting the ability for the original goals of the project to be as successful as they could be.

Re: The problem with OpenTelemetry

#79

Post by zeeg »

zellyn wrote: post #76

Are they basically just saying that the OpenTelemetry client APIs should be split from the rest of the pieces of the project, and versioned super conservatively? The simple API they describe is basically there in OTel. The API is larger, because it also does quite a few other things (personally, I think (W3C) Baggage is important too), but as a library author I should need only the client APIs to write to. When imple…

It’s a bit deeper than that. The SDKs that library authors implement need to be extemely minimal. The collection libraries that vendors implement based on imo should also be minimal.

OTLP imo doesn’t even need to be part of the spec.

But minimal would also mean focusing on solving fewer problems as a whole. Eg OpenTracing plus OpenMetrics plus OpenLogs. I only need one of those things.

Re: The problem with OpenTelemetry

#80

Post by brunoqc »

riedel wrote: post #67
brunoqc wrote: post #59

Earlier quoted context omitted.

But not foss. It's using the BSL or FSL or whatever.

Although I do not like those licences, I would not care so much about 2yrs until it goes FOSS. Before all this rush development RRDTool and OpenTSDB was so slow, this whole thing seems rather ideological than substantial criticism. Now going down the licence rabbit hole based to criticise the original argument seems like a classical strawman.

I was supporting a variation in my head of the "Yeah but who wants to contribute to an SDK for a service that you need to pay for?" claim.

You can self-host for free, so maybe @hahn-kev don't mind contributing to the SDK now.

For me, I refuse to contribute to an open-source SDK for a non-foss product. And I refuse to self-host a non-foss product.

Personally, I don't care if non-foss licenses speeds development. So yeah in my case it's ideological.

Post reply on HN