Live data from Hacker News

Why and how GitHub is adopting OpenTelemetry

github.blog

41–50 of 113 posts

Re: Why and how GitHub is adopting OpenTelemetry

#41
For everyone blindly rage triggered by the presence of the bytes "t-e-l-e-m-e-t-r-y", cogman10 elsewhere ITT [1] summarized that concern well:

> It's like being afraid your browser profiler is being used to spy on you. Could it do that? Sure. but there are so many easier ways to accomplish the same task.

So take a second to straighten out your panties and then actually look at the thing first. It's just an open-source (!) APM protocol that competes with (edit: more like, adjacent to) DataDog, DynaTrace, New Relic etc etc. It's not even a suitable tool for spying.

[1]: https://news.ycombinator.com/item?id=27296419

Re: Why and how GitHub is adopting OpenTelemetry

#42
post #11

Does this put "telemetry" in Git itself? If you're just using git to access Github, is it snooping on you?

No, OpenTelemetry is focussed on gathering metrics in distributed systems. Metrics, log aggregation and tracing of requests from the entrypoint (load balancer) all the way to the backend services and data stores. Take a look at the Jaeger and Zipkin websites and it should be pretty clear what it is used for.

And, it should be stated, that it isn't even really about gathering metrics. It is about providing a standard interface to gather metrics. (I know you know this, but with the confusion here, I figure I should go into details).

The point of OpenTelemitry is to make it so you could write the places you get your traces/metrics in one part of code and configure which backend system it is collected into in another part.

So, for example, you'd add a `trace("my slow thing"){ be slow }` into your code and later add `report to zipkin` or `report to Jaeger` in another part. The place where you trace "my slow thing" doesn't care about how to interact with the backend system or which backend vendor is ultimately used. You can start using prometheus, zipkin, new relic, Jaeger, whatever, just so long as they have an OpenTelemetry adapter you are golden.

The analog is SLF4J in Java.

Re: Why and how GitHub is adopting OpenTelemetry

#43
post #7
post #2

Are there any examples of companies making a contractually binding pledge to never use any telemetry data for anything than improving the application/service... NEVER for creating marketing profiles or for surfacing ads?

They're not using "telemetry" as a euphemism for "shipping spyware to end users to record their actions on their own machine", in this case. This appears to be a very fancy log aggregation product that has also eaten logging/tracing related configuration deployment functionality.

There is the web-packages in Javascript that creates traces for click events etc. but also the track how much time requests took on the client-side. Or render times of the UI e.g. Vue/React components

See: https://github.com/open-telemetry/opentelemetry-js-contrib/t...

Re: Why and how GitHub is adopting OpenTelemetry

#44

For everyone blindly rage triggered by the presence of the bytes "t-e-l-e-m-e-t-r-y", cogman10 elsewhere ITT [1] summarized that concern well: > It's like being afraid your browser profiler is being used to spy on you. Could it do that? Sure. but there are so many easier ways to accomplish the same task. So take a second to straighten out your panties and then actually look at the thing first. It's just an open-sourc…

Doesn't even compete with them :) Each of those have opentelemetry adapters.

You'd write your opentelemitry traces throughout your code and at some top level point in your app you configure and say "Hey, OpenTelemetry, report to New Relic".

By itself, OpenTelemetry does nothing.

It's usefulness is that someone writing a lib can add OpenTelemetry calls throughout and anyone using that lib can then collect metrics/traces into whatever metrics solution they are currently usings (be it DynaTrace, DataDog, or New Relic).

Re: Why and how GitHub is adopting OpenTelemetry

#45

For everyone blindly rage triggered by the presence of the bytes "t-e-l-e-m-e-t-r-y", cogman10 elsewhere ITT [1] summarized that concern well: > It's like being afraid your browser profiler is being used to spy on you. Could it do that? Sure. but there are so many easier ways to accomplish the same task. So take a second to straighten out your panties and then actually look at the thing first. It's just an open-sourc…

It doesn't really compete Datadog, Dynatrace, Honeycomb, Lightstep, new Relic are all partners in defining the specification and help implementing it. Most of them already allowing ingesting traces over the OLTP protocol

Re: Why and how GitHub is adopting OpenTelemetry

#46

Off topic: At some point my wasm app was consuming excessive memory bringing the browser to a halt. The issue was the OpenTelemetry package(Go-lang) used by various Google sdks. Forking/sanitizing the sdks fixed the issue.

What do you mean by “sanitizing” the sdk’s? Did you remove the OpenTelemetry package?

Re: Why and how GitHub is adopting OpenTelemetry

#47

For everyone blindly rage triggered by the presence of the bytes "t-e-l-e-m-e-t-r-y", cogman10 elsewhere ITT [1] summarized that concern well: > It's like being afraid your browser profiler is being used to spy on you. Could it do that? Sure. but there are so many easier ways to accomplish the same task. So take a second to straighten out your panties and then actually look at the thing first. It's just an open-sourc…

I tend to think computing itself has been in kind of a dumb rut. Right now, OpenTelemetry is just logging & tracing, imagined as tools for ops, but over time, I fully expect we begin to see this as an event-stream in itself, something we can use for Event Sourcing, to watch & trigger new compute based off of.

That's not in the cards today. But longer term, I think "knowing what computers are doing" is big business. And, I am very sad to say, eventually it will the obvious & logical way to spy on folks too. Because it will be the obvious & logical way to do many many many things, not because it's tech that's built or intended for spying. But right now computing is ephemeral, we don't persist any of the stack traces we compute through, and fundamentally, tracing really is about distilling out & keeping higher level stack traces. It's something computing needs to have been doing, that will bring us to a radically higher level of understanding (but it also does have some scary uses).

It's been a couple years, but tracing is a powerful new basis with which to start re-engaging the "Turning the datqbase inside out"[1] / "I [1] https://martin.kleppmann.com/2015/11/05/database-inside-out-...

Re: Why and how GitHub is adopting OpenTelemetry

#48

For everyone blindly rage triggered by the presence of the bytes "t-e-l-e-m-e-t-r-y", cogman10 elsewhere ITT [1] summarized that concern well: > It's like being afraid your browser profiler is being used to spy on you. Could it do that? Sure. but there are so many easier ways to accomplish the same task. So take a second to straighten out your panties and then actually look at the thing first. It's just an open-sourc…

It's interesting how names can send signals without any substance of the actual product being relevant. (The thread over 'bro pages' was one example.) This specific instance is rather revealing of one of the biases of some HN users.

I look at myself and I find that many opinions on technology that I hold have clearly been shaped by how much time I've spent on HN. Data-exfiltration telemetry is one example. But I now think about a lot of things I might not have given much thought to before, like how being backed by a VC can shape the direction of a service for the worse. I also find that I'm unnecessarily hardline on topics like paid services versus open source, and shut out anything positive people have to say about advertising as a revenue source.

I sometimes imagine what would happen if I was born inside the borders of a different country. Some countries have very nationalistic citizens. How much of my thought processes are a result of the people and signals I surround myself with? How many of those signals will reach me whether or not I consent to seeing them (advertisements being one example)?

I think I could have been a very different person if I was raised even a hundred miles from where I actually grew up, despite having a similar chemical makeup. That makes me be more conscious about where I'm likely to seek information, and to try not to look for opinions that I already agree with entirely.

Re: Why and how GitHub is adopting OpenTelemetry

#49
post #26

I think it would be great to show the performance impact of these SDKs because it is one of the really important aspects of monitoring (being non-intrusive).

Look into envoy/istio - e.g. these introduce side-processes (sidecars) where your process talks to, and these create the traces for you at some perf cost. There are proxies for some of the existing services - like mysql - https://www.envoyproxy.io/docs/envoy/latest/configuration/li... I haven't used it myself, I've used census, and now looking into OpenTelemetry (though from the least finished version - C++). Had mix…

envoy/istio do not replace telemetry in application because they only see what's pass through them and don't know anything else. You're missing a lot if you only instrument through a proxy.

Re: Why and how GitHub is adopting OpenTelemetry

#50
post #47

For everyone blindly rage triggered by the presence of the bytes "t-e-l-e-m-e-t-r-y", cogman10 elsewhere ITT [1] summarized that concern well: > It's like being afraid your browser profiler is being used to spy on you. Could it do that? Sure. but there are so many easier ways to accomplish the same task. So take a second to straighten out your panties and then actually look at the thing first. It's just an open-sourc…

I tend to think computing itself has been in kind of a dumb rut. Right now, OpenTelemetry is just logging & tracing, imagined as tools for ops, but over time, I fully expect we begin to see this as an event-stream in itself, something we can use for Event Sourcing, to watch & trigger new compute based off of. That's not in the cards today. But longer term, I think "knowing what computers are doing" is big business. A…

Who will be doing this spying?

It certainly wouldn't be opentelemetry itself as that's just a interface you add adapters to.

Are you thinking a man in the middle would spy? How would that work? This information is pushed over secure connections on the backend likely in a VPN. On the front end, it'd be transmitted over HTTPS. Shouldn't we be more fearful of information collected from DNS than encrypted data sent over HTTPS?

Or are you thinking the metrics aggregators are going to do the spying? How would that impact their business model? Do you think a company would continue to pay the likes of new relic if they were caught giving access to metrics data to outside groups? Do you worry about postgres or prometheus sharing your data with 3rd parties? What about SQL server?

What is the risk model and how would it be different from say the risk model of making an REST call or using a 3rd party library?

Or is it just that "because this is well integrated throughout, it could be used to spy"? Because, generally speaking, these traces don't have enough information to identify what individual users are doing to the system. Even if they did, that wouldn't be a great way to track a user, you'd simply put that tracking information right at the front end of the system. Plumbing it from one end of the system to the other gives little value for a spy. It's adding a bunch of noise to the question you'd want to ask "what are the user's behaviors with our product?"

And even if the demand is there, why would you do this through tracing an not a purpose built spy tool. Wouldn't it be easier for a nefarious lib writer to make a plugin purpose built to collect evil data? If that sells, why wouldn't a tech company buy that instead of buying a solution which combs 3 layers of separation to get worse answer? Why wouldn't google analytics still exist?

Post reply on HN