Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why and how GitHub is adopting OpenTelemetry

github.blog

Post reply on HN
Print view View original

31–40 of 113 posts

Re: Why and how GitHub is adopting OpenTelemetry

#31

Post by drewbug01 »

jeffbee wrote: post #19
StreamBright wrote: post #10

I think it would be great to show the performance impact of these SDKs because it is one of the really important aspects of monitoring (being non-intrusive).

I've noticed that orgs where I've worked vary between being totally insensitive to observability cost to being real hardasses about it. But I think most smaller shops are falling into the former category. I've even heard in meetings crazy shit like "It's very low overhead, only about 5%" which would get you laughed out of the office at, say, Google. Unfortunately (to me) the focus on ease-of-use has meant that OpenTe…

> Unfortunately (to me) the focus on ease-of-use has meant that OpenTelemetry concepts are structured in such a way to preclude even the possibility of a very efficient implementation

Curious what you mean about the design of OpenTelemetry precluding efficient implementation?

Re: Why and how GitHub is adopting OpenTelemetry

#33

Post by cogman10 »

infogulch wrote: post #28
handrous wrote: post #24

Earlier quoted context omitted.

"Telemetry" has come to be associated with activities that definitely qualify a product as spyware (though they've become fairly common—thanks, "data-driven" marketers). From reading the OpenTelemetry site, that doesn't seem to be their main, stated purpose, but thefounder's post in another part of the thread leads me to think it may, in fact, see that kind of use, too. [EDIT] damn, sorry, downvoters, for explaining…

Every tool is also a weapon, obviously. We haven't stopped making hammers because they can be used to hit people.

The sorts of stuff collected by opentelemitry are benign. Can it be weaponized? Of course, but that's like using a screwdriver to kill a fly. If I wanted to collect evil "telemitry" I wouldn't touch opentelemitry because it isn't useful for that.

It's like being afraid your browser profiler is being used to spy on you. Could it do that? Sure. but there are so many easier ways to accomplish the same task.

Re: Why and how GitHub is adopting OpenTelemetry

#34

Post by random3 »

AlexandrB wrote: post #20
rektide wrote: post #12

A lot of very confused weird comments here. This article is about a fairly large sized tech company adopting a fairly recent but increasingly mainstream & popular tool that helps them understand their operations. It'll give them a standard way to see what their computers are doing, across their various systems. OpenTelemetry is one of the key emerging cloud standards, and I expect many many many many more articles li…

> As for concerns about user tracking & privacy, that's not generally what these operational tools are used for. I'm not sure that the intended use matters. Unless the telemetry systems are carefully designed to not capture PII at all they become yet another channel that must be secured as if they are collecting PII. For example, see Windows 10 telemetry vs. HIPPA: https://hipaaone.com/2015/09/22/windows-10-and-hipaa…

They are agnostic data collection protocols/ systems / libraries. Just like a protocol they could do PII collection and unlikely to prevent that.

Ultimately the intended use matters more than the tech. And in this case it seems purposely built to measure performance.

Re: Why and how GitHub is adopting OpenTelemetry

#35

Post by cogman10 »

handrous wrote: post #30
infogulch wrote: post #28

Earlier quoted context omitted.

Every tool is also a weapon, obviously. We haven't stopped making hammers because they can be used to hit people.

It's why people are making these assumptions about the term "telemetry". In certain circles it usually means spyware. One of those circles includes front-end web, which is pretty well-represented on HN. [EDIT] and, especially, that's why they're jumping to the conclusion that github is gearing up to do more spying on its users, which is the part that I think people are bothered by, not the existence of this software…

OpenTelemitry isn't useful for spying purposes. It just isn't. If github wanted to spy on users they have way more direct methods that don't involve collecting the time your browser executes a function or waits on an AJAX request.

Re: Why and how GitHub is adopting OpenTelemetry

#36

Post by handrous »

cogman10 wrote: post #35
handrous wrote: post #30

Earlier quoted context omitted.

It's why people are making these assumptions about the term "telemetry". In certain circles it usually means spyware. One of those circles includes front-end web, which is pretty well-represented on HN. [EDIT] and, especially, that's why they're jumping to the conclusion that github is gearing up to do more spying on its users, which is the part that I think people are bothered by, not the existence of this software…

OpenTelemitry isn't useful for spying purposes. It just isn't. If github wanted to spy on users they have way more direct methods that don't involve collecting the time your browser executes a function or waits on an AJAX request.

Cool, that's fine, I was just explaining why it's getting that reaction from people. Telemetry means different things to different people and a (quite new) use is basically just a euphemism for "watching over people's shoulder while they use our software"—but, among some people, that's the main use they encounter, so they assumed that's what this is.

Re: Why and how GitHub is adopting OpenTelemetry

#37

Post by pm90 »

jeffbee wrote: post #19
StreamBright wrote: post #10

I think it would be great to show the performance impact of these SDKs because it is one of the really important aspects of monitoring (being non-intrusive).

I've noticed that orgs where I've worked vary between being totally insensitive to observability cost to being real hardasses about it. But I think most smaller shops are falling into the former category. I've even heard in meetings crazy shit like "It's very low overhead, only about 5%" which would get you laughed out of the office at, say, Google. Unfortunately (to me) the focus on ease-of-use has meant that OpenTe…

You're correct that 5% increase in resource usage is probably not noticeable for most orgs.

Its important to know what audience you're building for. I believe the audience for otel consists largely of companies that don't look anything like Google. So its fine to sacrifice that last bit of perf gain if it means the code is easier to use and maintain.

FWIW, it also appears that companies like Google would fork or reimplement such systems anyway.

Re: Why and how GitHub is adopting OpenTelemetry

#38

Post by cogman10 »

handrous wrote: post #7
mikece wrote: post #2

Are there any examples of companies making a contractually binding pledge to never use any telemetry data for anything than improving the application/service... NEVER for creating marketing profiles or for surfacing ads?

They're not using "telemetry" as a euphemism for "shipping spyware to end users to record their actions on their own machine", in this case. This appears to be a very fancy log aggregation product that has also eaten logging/tracing related configuration deployment functionality.

It's not even that. It's the interface that other logging/tracing apis can plug into. It's the "SLF4J" of tracing/metrics gathering.

Why is this useful? Because it lets people writing libraries provide the ability for users of their libraries to track performance information without dictating to them what performance tracking tools they should use.

Re: Why and how GitHub is adopting OpenTelemetry

#39

Post by wdb »

It's a really nice solution. I am using it to send traces and metrics via a standardised API in Go, and Node.js. During development I am sending without sampling it to Jaeger but in production I am sending ratio sampled traces to Google Trace but you can use Honeycomb or Lightstep. For metrics, I am sending them to local Prometheus during development, and cluster prometheus in production.

It's nice to see all the traces through all the different services of one API request and all its log items it generated. Using a custom span exporter to ensure PPI is stripped away as much as possible for span attributes.

Using W3C trace context and baggage to send some more data along, e.g. sending the trace id, span id etc into pub sub events to ensure to connect them with the trace.

Loving it! For now, I think it's worth the costs of adding tracing to services.

Re: Why and how GitHub is adopting OpenTelemetry

#40

Post by legulere »

jeffbee wrote: post #19
StreamBright wrote: post #10

I think it would be great to show the performance impact of these SDKs because it is one of the really important aspects of monitoring (being non-intrusive).

I've noticed that orgs where I've worked vary between being totally insensitive to observability cost to being real hardasses about it. But I think most smaller shops are falling into the former category. I've even heard in meetings crazy shit like "It's very low overhead, only about 5%" which would get you laughed out of the office at, say, Google. Unfortunately (to me) the focus on ease-of-use has meant that OpenTe…

It’s also about where your costs are. Google has much less revenue per compute-time/requests/whatever than most other companies. If you target the business to business field your computing costs are usually negligible while your developers cost a lot of money. Throwing more hardware at the problem is often the most economical solution.
Post reply on HN