Live data from Hacker News

Transparent telemetry for open-source projects

research.swtch.com

161–170 of 314 posts

Re: Transparent telemetry for open-source projects

#161

> the vast majority of projects, even large ones that would benefit, stay away from telemetry. Nomad is one of these projects. We support a dizzying array of platforms (32bit Intel Linux?!). We have no idea how popular our Consul service mesh integration is. Are bug reports a sign of use or just failed experiments? Is anyone running on macOS in production or just ephemeral dev agents? Surveys about this are just aski…

This is an incorrect assertion.

We have to ask for permission on our SaaS products to collect this data as it's not necessary to collect it for the product to function. The EU GDPR mandates this.

Russ Cox is suggesting that there is no permission step and that the data is collected by default.

That is the issue.

Re: Transparent telemetry for open-source projects

#162

Earlier quoted context omitted.

> IMHO if you want to collect any information, it should never be anything but opt-in, a conscious decision. Serious (general) question: How do you do that given a non-technical user population? Debian’s opt-in popcon kind of manages to get a little bit of data from a fairly technical one, but nowhere near enough to estimate a low usage frequency, and it’s the only opt-in program I’m aware of that gets anything usabl…

How many people are installing and using open source software but couldn't understand a pop-up explaining what data is collected and asking if they'd like to submit it? Is the non-technical nature of the user the problem or is it just that when you have an opt-in option most people make the choice to opt-out? That's the thing about respecting users by giving them choice, they get to say no. If they mostly say no, and…

> How many people are installing and using open source software but couldn't understand a pop-up explaining what data is collected and asking if they'd like to submit it?

I’ve taught probability theory using randomized response[1] as an exercise problem, and while people can understand it given time and motivation, it’s not immediately obvious. So I’m not exactly hopeful that a prospective Audacity, Blender, or even Free Pascal user (to take an arbitrary set of examples) would get what I mean if I say “I’m collecting no more than 10 bits of information about you using RAPPOR”[2], and I’m not willing to engage in comforting bullshit such as “all collected data is anonymous”, as I’ve been all too close to situations where the difference between the two might be one between freedom and prison.

> Is the non-technical nature of the user the problem or is it just that when you have an opt-in option most people make the choice to opt-out?

Both, because confirmation dialogs, especially privacy-related ones, have been thoroughly poisoned in users’ minds. But confirmation of obscure actions, however beneficial their consequences, is problematic in general—if I go on the street and ask people if they’d like caffeine in their tea or ascorbic acid in their apples, I expect (but have not checked) that the majority will say no, nevermind that both are normally there and intrinsic to the experience.

(The possibility of meaningful consent from a non-specialist is the subject of much discussion and few good answers in med school, or so I’ve heard.)

Whether the ultimate answer is to grant or deny permission, I’m not sure I can present the question in a way that will actually have it made on the basis of merit and not on “scary permission dialog, better say no” or “yes, yes, just let me through to my dancing bunnies[3]” or “yes, if I say no the installer will just tell me to GTFO”.

(In that respect the “Send crash report to vendor” button is unexpectedly good, because you’re not actually interposing yourself between the user and any prospective bunnies. But personally I don’t like to spend time and effort in order to send “feedback” into an unmarked hole where I’ve no idea if anybody will ever look at it. From that point of view, it is background data collection that’s unexpectedly good.)

And even if, for the purposes of this question, it would be best if people took the time to learn the necessary maths, computing, and operational security to make an informed choice, in reality I’m not sure that’s the best thing they can spend their life on.

So it may be the answer is that you simply can’t do telemetry well for the social reason that users won’t ever end up making an informed choice, or that the well has been poisoned so thoroughly that the rational choice is to reject everything. It’s just that I know that it’s basically possible in a technical sense, so I don’t want to give up that easily.

[1] https://en.wikipedia.org/wiki/Randomized_response

[2] https://blog.cryptographyengineering.com/2016/06/15/what-is-...

[3] https://blog.codinghorror.com/the-dancing-bunnies-problem/

Re: Transparent telemetry for open-source projects

#163
post #159

@rsc, if you ever see this, your proposal here means that I will never use any software written in Go ever again, if at all possible. What others have said in this thread about telemetry becoming an "accelerant" will happen. Abuse will happen. Data will be put up for sale. IP's will be logged because users can't verify that they're not. The only thing users can verify is what is sent and to whom. And only if they run…

What world do you live in?

All browser have telemetry in them, but he probably used netcat to post that message.

Re: Transparent telemetry for open-source projects

#164

Probably related to[0]. To anybody complaining that this should be opt-in: opt-in telemetry doesn't work. The reason for this is that most people don't care, but they don't care either way. They don't disable it when prompted, nor would they enable it manually. The idea of telemetry is being able to prioritize the work that will be most widely useful. For this you need a good and balanced sample of your users. You do…

But that's wrong. There is no position for this in a civilised society:

"If we ask everyone is going to say no, so we will steal it unless someone tells us not to"

Re: Transparent telemetry for open-source projects

#165
There's a lot of strong reactions here, which I don't think are generally unfounded. Telemetry has certainly been misused and will continue to be, but it can also be an invaluable tool for product development.

For example, we had a CLI with many commands and flags, some of which were costly to maintain. By adding analytics, we were able to see that literally no one was using certain commands, and we could safely remove them without messing up workflows.

On each CLI invocation, we collected:

  - hash of user ID
  - which command is run
  - which flags were included
  - operating system (not version information, just mac/pc/linux)
This data wasn't used for marketing, had no identifiable information, and was diasablable (but opt-out). You could also log exactly what was sent to the server, so you could see.

We could have collected some of this via occasional surveys, but the data would have been less useful and less accurate.

I didn't look into the details of what Go is proposing to collect, but treating all telemetry of any kind as a boogeyman isn't productive; just have to do it the right way.

Re: Transparent telemetry for open-source projects

#166
post #42

Earlier quoted context omitted.

> IMHO if you want to collect any information, it should never be anything but opt-in, a conscious decision. Serious (general) question: How do you do that given a non-technical user population? Debian’s opt-in popcon kind of manages to get a little bit of data from a fairly technical one, but nowhere near enough to estimate a low usage frequency, and it’s the only opt-in program I’m aware of that gets anything usabl…

What I hear you saying here is that people don't do what you want if you give them the choice, so you lean towards not giving them the choice rather than respecting their wishes. Is my interpretation correct?

>> I’m unwilling to implement an opt-out system

> [Y]ou lean towards not giving them the choice rather than respecting their wishes.

> Is my interpretation correct?

I don’t think it is, no :) Rather, I’m not sure how to sell, to put it crassly, users on a choice when properly investigating or even being confronted with that choice would delay them seeing the dancing bunnies[1], but that would also, if I have any say about it, improve the bunnies in the future.

Does that mean there’s a shade of “I know better” in my problem statement? Of course it does, if I didn’t know better than the average user I’d have no business designing such choices. I don’t think there’s anything wrong about that, better than the average at an activity few practice is not a terribly high bar. Not giving the users a choice or manipulating them into making the one I think is right would absolutely be wrong, though.

Basically, how do I make the user think, how do I give them the appropriate data to do so, and how do I deal with the obvious contradiction of that goal with principles of good design[2]? The potential benefits to the software and (thereby) the users are too much to give up without even asking those questions.

(See nearby comment for extended discussion.)

[1] https://blog.codinghorror.com/the-dancing-bunnies-problem/

[2] https://sensible.com/dont-make-me-think/

Re: Transparent telemetry for open-source projects

#167
post #112

Earlier quoted context omitted.

What is the argument for opt-out telemetry being unethical?

Because that goes against informed consent. Opt-out is generally rejected by European privacy laws.

> Opt-out is generally rejected by European privacy laws.

...where personal data is involved.

It strikes me that this proposal goes to considerable lengths to avoid collecting anything that could be considered personal data.

Re: Transparent telemetry for open-source projects

#168
post #42

Earlier quoted context omitted.

What I hear you saying here is that people don't do what you want if you give them the choice, so you lean towards not giving them the choice rather than respecting their wishes. Is my interpretation correct?

>> I’m unwilling to implement an opt-out system > [Y]ou lean towards not giving them the choice rather than respecting their wishes. > Is my interpretation correct? I don’t think it is, no :) Rather, I’m not sure how to sell, to put it crassly, users on a choice when properly investigating or even being confronted with that choice would delay them seeing the dancing bunnies[1], but that would also, if I have any say…

Thank you for the thoughtful response. We disagree on much, but I respect your opinion nonetheless.

> Not giving the users a choice or manipulating them into making the one I think is right would absolutely be wrong, though.

I'll pull out just this point, though, to perhaps illustrate how different our worldviews are. I consider opt-out to be a manipulative approach.

Re: Transparent telemetry for open-source projects

#169

Earlier quoted context omitted.

Reaching a different conclusion is one thing, but not seeing a dilemma is another. One can always argue that invading a person's autonomy might be necessary given the benefits but seeing no issue is just turning a blind eye.

In the Golang announcements, it's clear that they completely see and understand the dilemma, and have provided a lengthy explanation of why they decided for opt-out anyway. I respect that. I don't agree with the decision, but it was made with understanding and thought.

I made my original comment misunderstanding what the parent comment meant as "not knowing an ethical problem exists". I also am not talking about this specific decision, but criticizing ethical decision making in the tech industry in general.

In ethics, there is no right or wrong answers (mostly), just right and wrong methodologies. If you go the pragmatic way, you'd argue that the benefits of telemetry are greater than the downsides and implement it. If you go Kant's way, you would already have a maxim (either "never invade privacy" or "prioritize technical benefits regardless of the users" in this case) and act according to that maxim regardless of the situation. If you go the intent way, all that matters is whether your intent for the action is good or bad, in contrast if you go the outcome way, all that matters is the outcome regardless of the intent or the methods.

These are all "valid" ways to discuss an ethical dilemma. However, one must always acknowledge the dilemma. This industry, especially big tech, seems to ignore this quite often, mostly because it's very easy to see people as "just numbers" when you don't see them directly. Don't even get me started on lawmakers who are also ignoring this whole issue. Many standard practices in this industry would be straight up illegal in lots of other areas, especially where there is face-to-face contact.

Re: Transparent telemetry for open-source projects

#170

Earlier quoted context omitted.

>> I’m unwilling to implement an opt-out system > [Y]ou lean towards not giving them the choice rather than respecting their wishes. > Is my interpretation correct? I don’t think it is, no :) Rather, I’m not sure how to sell, to put it crassly, users on a choice when properly investigating or even being confronted with that choice would delay them seeing the dancing bunnies[1], but that would also, if I have any say…

Thank you for the thoughtful response. We disagree on much, but I respect your opinion nonetheless. > Not giving the users a choice or manipulating them into making the one I think is right would absolutely be wrong, though. I'll pull out just this point, though, to perhaps illustrate how different our worldviews are. I consider opt-out to be a manipulative approach.

> I consider opt-out to be a manipulative approach.

So do I, which is why I wrote I’m unwilling to implement it :) The original (and, to be clear, purely theoretical) point was, opt-out is too manipulative while opt-in is likely useless.

Ah shoot. Did you take that to mean that I’m unwilling to implement an off switch at all? That wasn’t it, sorry for the confusion.

Post reply on HN