Live data from Hacker News

Transparent telemetry for open-source projects

research.swtch.com

21–30 of 314 posts

Re: Transparent telemetry for open-source projects

#21

Honestly, this may be unpopular with hacker news, but just add your own telemetry. If people don't like it they can turn it off, and telemetry is essential for a good product. Do let people turn it off though please.

> telemetry is essential for a good product

Up until now, you've had to make these design decisions on your own, relying only on perplexing intangibilities like 'taste' and 'intuition'.

Re: Transparent telemetry for open-source projects

#22
Telemetry in open source exists for a long time. Debian has the popcon package that can be installed and reports weekly usage of the software packages. The telemetry data are published in the open. The Debian popcon FAQ could be used as guideline for other telemetry needs. https://popcon.debian.org/

Re: Transparent telemetry for open-source projects

#23
I've been a pretty strong advocate of the idea that analytics should always be minimal, 100% anonymous, aggregated, and open to the public - otherwise it’s spying. This is how we do analytics on our websites today[0][1], and how we plan to do it in games we release in the future. Maybe one day I will start a dedicated FOSS service that people can use for exactly this with some trusted reputation/transparency/auditability to it.

I think what Russ has described here is decent and well-reasoned. I also think that Go being a product (it is, whether you like that word or not) makes it more fair to desire analytics of this form. I think it being opt-out is reasonable (after all, if it is not, they will make decisions using data that does not come from the vast majority of users, may as well not have analytics at all then.)

But I am afraid of this becoming pervasive not just in products (like CLI tools), but also in libraries, imagine every Go/npm package you use wants to ping the network because the authors want to know 'is this popular? can we deprecate XYZ method?' etc. If transparent telemetry in the form Russ and I have been viewing it becomes a more common thing, it won't be a surprise if more library authors begin to try to adopt something like this and it becomes a pervasive problem IMHO.

[0] https://hexops.com/privacy

[1] https://machengine.org

Re: Transparent telemetry for open-source projects

#24
This is a good plan, very simple and clear, and I like the list of system properties at the end. The solution is pretty tailored for the Go toolchain, which is a good strategy that has worked for them in the past.

A more general purpose metrics tool I'm watching closely is Divvi Up https://divviup.org/, a research project by ISRG, the same org that runs LetsEncrypt. The basic idea is to divide up each metric into two parts and publish each part to separate collection servers (one run by you and the other by divviup). Then the servers separately aggregate their half and combine the results, the idea being that each half is useless on it's own but when combined it's still useful.

I wouldn't suggest it for this application, but for the majority of typical apps it would be a vast improvement to privacy compared to the status quo.

Re: Transparent telemetry for open-source projects

#25
post #18

I wish there was a standard way of disabling telemetry across software dependencies. While I leave it turned on for personal projects, several projects at work require disabling it. I have spent hours auditing through transitive dependencies to turn it off. It should not be this painful.

No reason they couldn't all aim to respect a `TELEMETRY=false` env var, akin to the web's 'do not track' request.

Re: Transparent telemetry for open-source projects

#26

> the vast majority of projects, even large ones that would benefit, stay away from telemetry. Nomad is one of these projects. We support a dizzying array of platforms (32bit Intel Linux?!). We have no idea how popular our Consul service mesh integration is. Are bug reports a sign of use or just failed experiments? Is anyone running on macOS in production or just ephemeral dev agents? Surveys about this are just aski…

Everything involves tradeoffs.

The times "we" (previous companies) tried to implement telemetry in open source non-SaaS products (as distinct from "projects"), we either got huge blowback or users/customers simply blocked it at the firewall (and security teams at major enterprises were unwilling to open holes anyway).

The only workable solution I found was integrating this in a value-add way, so that something in the service/experience/etc was better for the user/customer as a result of enabling telemetry, without the dark pattern of making things intentionally awful/worse without it. We simply never got enough data to matter otherwise. But, again, that was products and not projects.

Re: Transparent telemetry for open-source projects

#27
post #23

I've been a pretty strong advocate of the idea that analytics should always be minimal, 100% anonymous, aggregated, and open to the public - otherwise it’s spying. This is how we do analytics on our websites today[0][1], and how we plan to do it in games we release in the future. Maybe one day I will start a dedicated FOSS service that people can use for exactly this with some trusted reputation/transparency/auditabi…

I am concerned about run-time telemetry in libraries as well. It might make sense for language ecosystems to offer more data about library usage gathered at build time eventually, as a different system than the one I'm posting about today. I think when you get to that level of detail you probably need to start thinking hard about differential privacy and probably cryptographic solutions like ESA or Prio. I don't think we know enough to design the library solution yet.

Re: Transparent telemetry for open-source projects

#28

> the vast majority of projects, even large ones that would benefit, stay away from telemetry. Nomad is one of these projects. We support a dizzying array of platforms (32bit Intel Linux?!). We have no idea how popular our Consul service mesh integration is. Are bug reports a sign of use or just failed experiments? Is anyone running on macOS in production or just ephemeral dev agents? Surveys about this are just aski…

On the contrary, I'd argue that the tracing visibility you're looking at isn't inherently a software trait at all. It's a deployment feature, which is something you address at-cost when building a product, but almost never when building FOSS software. It's not that people in FOSS don't see that upsides to it, it's that those upsides are insignificant relative to the cost of sustained market research. It's easier to just... make stuff, and have companies plaster over the gaps when their interests align.

Look at GNOME, which recently pushed for it's users to contribute telemetry: https://linuxiac.com/gnome-survey-results/

Nothing wrong with what they've done here, but we already had most of these metrics. Nothing was really learned, and it took Red Hat and a few thousand users to get here. For smaller-scale projects, imagine how much smaller the returns would be.

Re: Transparent telemetry for open-source projects

#29
post #25
post #18

I wish there was a standard way of disabling telemetry across software dependencies. While I leave it turned on for personal projects, several projects at work require disabling it. I have spent hours auditing through transitive dependencies to turn it off. It should not be this painful.

No reason they couldn't all aim to respect a `TELEMETRY=false` env var, akin to the web's 'do not track' request.

Does anyone really respect DNT though?

Re: Transparent telemetry for open-source projects

#30
post #19

Honestly, this may be unpopular with hacker news, but just add your own telemetry. If people don't like it they can turn it off, and telemetry is essential for a good product. Do let people turn it off though please.

> If people don't like it they can turn it off If I, perchance, encounter software I use phoning home without my explicit permission it's done on my systems. Period.

That is fine, but in this case telemetry trades you (and other more hardline users) as a user for all the extra users you gain from instant crash reports, quick feedback, and generally better productivity.

I would never personally make that trade-off, and would always put (disableable) telemetry in my projects.

Post reply on HN