Live data from Hacker News

Transparent telemetry for open-source projects

research.swtch.com

241–250 of 314 posts

Re: Transparent telemetry for open-source projects

#241
post #195

Earlier quoted context omitted.

You're framing this as though the "ethical" choice were obvious, or that there was a person who "knew this was the ethical thing to do, but turned a blind eye". I disagree, I think it's a very contested topic, with lots of discussion whenever it's raised here, with either side possibly being a vocal minority.

Spying on people without their consent is not ethical.

How would you use compiler telemetry to spy on people?

Re: Transparent telemetry for open-source projects

#242
post #161

> the vast majority of projects, even large ones that would benefit, stay away from telemetry. Nomad is one of these projects. We support a dizzying array of platforms (32bit Intel Linux?!). We have no idea how popular our Consul service mesh integration is. Are bug reports a sign of use or just failed experiments? Is anyone running on macOS in production or just ephemeral dev agents? Surveys about this are just aski…

This is an incorrect assertion. We have to ask for permission on our SaaS products to collect this data as it's not necessary to collect it for the product to function. The EU GDPR mandates this. Russ Cox is suggesting that there is no permission step and that the data is collected by default. That is the issue.

GDPR only cover PI data so your comment is irrelevant.

https://gdpr.eu/eu-gdpr-personal-data/

Re: Transparent telemetry for open-source projects

#243

Nope, nope, and more nope. You're not moving the Overton Window any more on me. In fact it seems there's a clear correlation between the quality of software and how much spyware there is embedded in it. It's often merely another way to justify unpopular changes with "but the data says so". IMHO if you want to collect any information, it should never be anything but opt-in, a conscious decision.

So how would you design a well-working system to make data-driven decisions rather than guesses? Most collection methods are notoriously bad, but partial collection is also bad since we now have to somehow put a weighing factor on presumed absent data, which turns choices into guesses again. I think this is a really hard problem, and simply trying to guess in the dark as to what people want isn't the smartest way to…

The whole idea of "data-driven decisions" is the problem.

It's an excuse to not respect user's choices and absolve oneself of the blame, or regress to a lowest-common-denominator, "because the data says so".

"Not everything that counts can be counted, and not everything that can be counted, counts."

Re: Transparent telemetry for open-source projects

#244

Earlier quoted context omitted.

Free disk space can be as low as zero, but we don't blame the tool makers for adding an extra 100Kb or 20MB, we blame the computer owner for not having enough disk space to install the thing they chose. Wrapper scripts for every utility to do UTIL_TELEMETRY_OPT_OUT=1 util ... so they don't need to be set all at once.

> we don't blame the tool makers for adding an extra 100Kb or 20MB Honestly, I do. Code bloat is a real thing.

Honestly, I do too, but the world doesn't. If you said "I have 3,500 binaries on my system, imagine if they ALL added 1MB" the reply would be "3.5GB is about twenty cents of NVME storage space" not "oh my, you're right that would be intolerable".

Re: Transparent telemetry for open-source projects

#245

  The Go team at Google would run a collection server. Each week, with 10% probability (averaging ~5 times per year) the user’s Go installation would download a “collection configuration” to find out which counter values are of interest to the server and at what sample rate.
If there's interest to use config files to determine how telemetry is done, why can't similar be done about turning telemetry off? I don't want to deal with environment variables (for a gazillion reasons) and would prefer to just use a config file. Especially when it comes to sending arbitrary information from my system to another arbitrary host.

It's so strange to me that the configuration of telemetry has been escalated to uses-configs status while opting out hasn't. Really feels like opting out is an after thought.

Re: Transparent telemetry for open-source projects

#246
post #73

Earlier quoted context omitted.

The argument for this being opt-in isn't about "it works better", it is about it being ethically correct. There are a ton of things that "don't work" unless you do something unethical: that doesn't mean they are OK, it doesn't mean they should be tolerated, and it doesn't mean the people who do them--and, at the end of the day, it is people who make these decisions: there is a human being who refused to say "no" and…

You're framing this as though the "ethical" choice were obvious, or that there was a person who "knew this was the ethical thing to do, but turned a blind eye". I disagree, I think it's a very contested topic, with lots of discussion whenever it's raised here, with either side possibly being a vocal minority.

The ethical choice is obvious.

The distinction is between "What I do with my computer is none of your business unless I choose to make it your business" versus "What I do with my computer is your business unless I choose to not make it your business".

It's insane that we are still having to justify privacy as a default, or that people continue to rationalize away the concerns.

Yeah, maybe if it's opt-in they won't have much telemetry data. Perhaps, in fact, it would not be much better than having no data at all. That will make some things harder. Major bummer. If it was easy to do the right thing, more companies would do it.

Re: Transparent telemetry for open-source projects

#247
post #245

The Go team at Google would run a collection server. Each week, with 10% probability (averaging ~5 times per year) the user’s Go installation would download a “collection configuration” to find out which counter values are of interest to the server and at what sample rate. If there's interest to use config files to determine how telemetry is done, why can't similar be done about turning telemetry off? I don't want to…

As designed, the system allows an opt-out _either_ by setting GOTELEMETRY=off in your environment or by running 'go env -w GOTELEMETRY=off' which writes a config file. (Specifically the one reported by 'go env GOENV'.) If you prefer to edit the config file directly, you are of course welcome to do that.

Re: Transparent telemetry for open-source projects

#248

Earlier quoted context omitted.

I think maybe even making telemetry mandatory with an open license, and customizable with a support license might be a sustainable way to run an open source company. For many open source project (anything with an attached business model), either telemetry or tight communication around usage patterns will be necessary to inform development. The latter of those two options consumes business resources.

> making telemetry mandatory with an open license If it's mandatory to run the code that does telemetrics, it's not a very open license.

Just because Linux is open source doesn't mean you can't have both Fedora and Red Hat (an enterprise version built on the same codebase)

I don't think any closed source goes into Red Hat, it's just the patch delivery pipelines, package repositories, etc that require a license. And support of course.

Same with any distributed system whose core contributors could gain insight from telemetry. All the components are open source, but they can package it all up and make it available under different terms.

If there's a community version and an enterprise version, you can then make telemetry required in the community version. If people don't like it, they can pull the package apart and put it back together however they want, or they can pay for an enterprise license.

Re: Transparent telemetry for open-source projects

#249
post #218

Earlier quoted context omitted.

> IMHO if you want to collect any information, it should never be anything but opt-in, a conscious decision. Serious (general) question: How do you do that given a non-technical user population? Debian’s opt-in popcon kind of manages to get a little bit of data from a fairly technical one, but nowhere near enough to estimate a low usage frequency, and it’s the only opt-in program I’m aware of that gets anything usabl…

First, we are talking about Development tools, so not non-technical population. Second, if Opt-in is considered difficult for the population, what does it say about the opt-out? Opt-out is always, no exception, more difficult than opt-in.

Seems like you also[1] didn’t read the above the way I intended. I meant that I find explicit opt-out (as opposed to explicit opt-in) manipulative so I don’t want to implement it, not that I oppose having the ability to opt out at all.

The difficulty, though, lies not (entirely) with the default position of the toggle, the difficulty lies with making the user think about the question which is not relevant to their immediate task and which in any case they may not have the theoretical tools or time to evaluate properly. The default position of the toggle (if “off”, as I believe it should be) matters only because an opt-in process means you either confront the user with irrelevant questions on first launch or get essentially no data.

(I called this out as a a “general” question because I meant for it to apply not only to the Go toolchain, but to general-use software like Firefox or niche but non-programmer-oriented software like Audacity.)

The systemwide daemon proposed elsethread[2] would solve this nicely as well, but I have to admit that I’ve dismissed it from my thought process more than once before, because I didn’t think we were going to get one with any reasonable usage on any platform. Now that I’ve seen it put in writing, maybe it does deserve to be considered.

[1] https://news.ycombinator.com/item?id=34716342

[2] https://news.ycombinator.com/item?id=34709836

Re: Transparent telemetry for open-source projects

#250
post #247
post #245

The Go team at Google would run a collection server. Each week, with 10% probability (averaging ~5 times per year) the user’s Go installation would download a “collection configuration” to find out which counter values are of interest to the server and at what sample rate. If there's interest to use config files to determine how telemetry is done, why can't similar be done about turning telemetry off? I don't want to…

As designed, the system allows an opt-out _either_ by setting GOTELEMETRY=off in your environment or by running 'go env -w GOTELEMETRY=off' which writes a config file. (Specifically the one reported by 'go env GOENV'.) If you prefer to edit the config file directly, you are of course welcome to do that.

That's fair, but doesn't take away from the deeply, deeply exhausting need to do this sort of niche configuration thing for everything out there, let alone even recognizing that it's needed. Never mind the unduly need to read changelogs and blog posts to make sure things haven't changed since I last used things.

Have you considered using a more-generic opt-out environment variable that's not go specific? For example "USER_PREF_NO_TELEMETRY=true" would have the same effect, or would ensure that GOTELEMETRY=off is set. I have no idea if anything like that exists right now in other projects, but if it's not, then go is large enough and embedded across enough systems that it could be a good place to start.

Post reply on HN