Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Transparent telemetry for open-source projects

research.swtch.com

Post reply on HN
Print view View original

11–20 of 314 posts

Re: Transparent telemetry for open-source projects

#11

Post by geodel »

autoexec wrote: post #6

> Although the report would not include any identifiers, the TCP connection uploading the report would expose the system’s public IP address to the server if a proxy is not being used. This IP address would not be associated with the uploaded reports in any way. Any fully transparent data collection is going to have to include IP addresses and timestamps. Even if the IP isn't being used for debugging, the software st…

> their favorite open source projects collecting telemetry.

Their favorite Google open source project. This is specially important for project which can't realistically exist without main sponsor / benefactor. It also help people to pay whatever little/high cost in term of conscience when they take part or consume something willingly but do not approve of makers.

Re: Transparent telemetry for open-source projects

#12

Post by schmichael »

> the vast majority of projects, even large ones that would benefit, stay away from telemetry.

Nomad is one of these projects. We support a dizzying array of platforms (32bit Intel Linux?!). We have no idea how popular our Consul service mesh integration is. Are bug reports a sign of use or just failed experiments? Is anyone running on macOS in production or just ephemeral dev agents?

Surveys about this are just asking humans to do something computers can do better.

Obviously privacy and consent are paramount concerns, but not only are they solvable, in open source they’re fully auditable (and a fork could fairly easily maintain a patch that removes it outright).

I think open source largely rejecting telemetry puts it at a huge disadvantage to proprietary and SaaS software where it is the norm. I’m very excited to see someone as thoughtful and well reasoned as Russ Cox to be trying to move the status quo forward.

Re: Transparent telemetry for open-source projects

#13

Post by mordae »

I dunno. It sure makes sense to me to collect telemetry from free software installations, but I feel that having every platform or even piece of software to do it on its own with opt-out will inevitably lead to people being overwhelmed and angry.

I would, personally, prefer a single non-profit service that would list publicly what is being collected and publish the results as open data for anyone to use. Applications (at least on Linux) would not submit their reports directly, but would use a local relay service that could be turned off completely or that could filter what reports to send to the server and what to /dev/null.

Distributions and other software stores would then make it mandatory for software to use this relay and either patch out any other telemetry from their packages or straight out forbid those that would not comply.

Re: Transparent telemetry for open-source projects

#14

Post by mananaysiempre »

userbinator wrote: post #3

Nope, nope, and more nope. You're not moving the Overton Window any more on me. In fact it seems there's a clear correlation between the quality of software and how much spyware there is embedded in it. It's often merely another way to justify unpopular changes with "but the data says so". IMHO if you want to collect any information, it should never be anything but opt-in, a conscious decision.

> IMHO if you want to collect any information, it should never be anything but opt-in, a conscious decision.

Serious (general) question: How do you do that given a non-technical user population? Debian’s opt-in popcon kind of manages to get a little bit of data from a fairly technical one, but nowhere near enough to estimate a low usage frequency, and it’s the only opt-in program I’m aware of that gets anything usable at all. Given that I’m unwilling to implement an opt-out system, I don’t really see a workable approach here at all.

Re: Transparent telemetry for open-source projects

#15

Post by gen220 »

mordae wrote: post #13

I dunno. It sure makes sense to me to collect telemetry from free software installations, but I feel that having every platform or even piece of software to do it on its own with opt-out will inevitably lead to people being overwhelmed and angry. I would, personally, prefer a single non-profit service that would list publicly what is being collected and publish the results as open data for anyone to use. Applications…

I think the issue of telemetry is fundamentally a human issue of incentives and trust. The system you describe is wise because it recognizes this and attempts to address it.

The difficulty with telemetry is that even if we design the perfect, privacy-preserving system to begin with, once the pattern of having a network port open is established, there's nothing to prevent us (humans) from changing our policies about what we're allowed to push/pull over that port.

In real-world analogues for these kinds of thorny policy problems, we have centralized arbiters to solve these problems. That might be a fruitful course of research for people interested in this problem to explore.

Unfortunately, even though this problem has software as its medium, it is a problem that cannot be solved by clever software alone, despite any appearances to the contrary.

Re: Transparent telemetry for open-source projects

#17

Post by rsc »

mananaysiempre wrote: post #8
autoexec wrote: post #6

> Although the report would not include any identifiers, the TCP connection uploading the report would expose the system’s public IP address to the server if a proxy is not being used. This IP address would not be associated with the uploaded reports in any way. Any fully transparent data collection is going to have to include IP addresses and timestamps. Even if the IP isn't being used for debugging, the software st…

> Any fully transparent data collection is going to have to include IP addresses and timestamps. Even if the IP isn't being used for debugging, the software still phones home and the IP is still being collected and logged when it otherwise wouldn't be. Either when uploading the report or when downloading the “collection configuration”. How do you verifiably not collect users’ IP addresses when receiving data from the…

The only option is to have a proxy sit in the middle between the uploader and the server. You mentioned Tor but it doesn't have to be Tor, just some proxy most users would trust not to collude with the server and that doesn't itself derive benefit from seeing the IP addresses. If there were a different entity that could be relied upon to run servers doing this and were highly trusted by users, I'd be interested to use it. Failing that, the usual answer for an enterprise or company is to run their own HTTP proxy. The design explicitly supports that.

Re: Transparent telemetry for open-source projects

#18

Post by taveras »

I wish there was a standard way of disabling telemetry across software dependencies.

While I leave it turned on for personal projects, several projects at work require disabling it.

I have spent hours auditing through transitive dependencies to turn it off. It should not be this painful.

Re: Transparent telemetry for open-source projects

#19

Post by groestl »

bioemerl wrote: post #16

Honestly, this may be unpopular with hacker news, but just add your own telemetry. If people don't like it they can turn it off, and telemetry is essential for a good product. Do let people turn it off though please.

> If people don't like it they can turn it off

If I, perchance, encounter software I use phoning home without my explicit permission it's done on my systems. Period.

Post reply on HN