[dead]
GitHub CLI now collects pseudoanonymous telemetry
321–330 of 368 posts
Re: GitHub CLI now collects pseudoanonymous telemetry
#322Earlier quoted context omitted.
I used to believe that it was not necessary until I started building my own startup. If you dont have analytics you are flying blind. You don't know what your users actually care about and how to optimize a successful user journey. The difference between what people tell you when asked directly and how they actually use your software is actually shocking.
Analytics is wrong. I never click any ads, but they keep showing it. I avoid registering or enter fake emails, but they keep showing full screen popups asking for email. I always reject cookies but they still ask me to accept them. And youtube keeps pushing those vertical videos for alternately gifted kids despite me never watching them. What's the point of this garbage analytics. It seems that their only goal is to…
All of those are affected by analytics.
Ad slots will be filled whether or not you click. If you never click, you'll tend to match with either very low quality ads or ads that pay per impression (display ads).
Email registration is highly valuable for a business, so analytics won't be used to decide whether to show the modal but rather test different versions of it.
Cookies are too valuable to not push on users, because without them only the previously mentioned low quality ads can be shown. High quality and display ads match on interest or demographic labels.
The business decision to keep vertical videos is highly likely to be affected by analytics, and of course the choice of which videos to show is based on recommendation models trained on interaction logs.
The priority isn't making your experience better, though that is often an incidental result -- it's driving the business.
Re: GitHub CLI now collects pseudoanonymous telemetry
#323Earlier quoted context omitted.
The ground truth that I never click on Stargate on Netflix is completely at odds with the actual truth that I love Stargate and want more of it and things like it. What the ground truth usage data is completely ignorant of is that Netflix's copy is a crappy blurry transfer, and so I got dvds instead.
Telemetry doesn’t tell the “why”. You never clicking in Stargate in Netflix is apparently true, so the telemetry isn’t wrong. It just doesn’t answer why.
Le duh. The whole point is that the perfectly true data is misleading and uninformative, and the "ground truth usage data" argument has a plot hole.
It's not that it has no value at all, it's just that it's stupid to know one thing (how to collect usage data) and think that is all you need to know and that that obviates all other sources of understanding.
If you wish to collect money from human customers, you have to be some minimum level of human yourself. Talking to your customers is not some icky hardship to be avoided and replaced with nice bash script.
Re: GitHub CLI now collects pseudoanonymous telemetry
#324Earlier quoted context omitted.
The ground truth that I never click on Stargate on Netflix is completely at odds with the actual truth that I love Stargate and want more of it and things like it. What the ground truth usage data is completely ignorant of is that Netflix's copy is a crappy blurry transfer, and so I got dvds instead.
Sure, but Netflix is not interested in whether you love Stargate or not. Telemetry says that you never click it, so it's ok to remove it from their catalogue (which is correct). Now, they could've done a better job by increasing the quality, but that's a further (and costly) optimisation.
It's not correct. I paid someone else for dvds. A little more of that and I may consciously question why I pay both netflix and ebay.
Re: GitHub CLI now collects pseudoanonymous telemetry
#325Earlier quoted context omitted.
You're only flying blind if you make decisions not looking and thinking. Analytics isn't the only way to figure out "what your users actually care about", you can also try the old school way, commonly referred to as "Talking with people", then after taking notes, you think about it, maybe discuss with others. Don't take what people say at face value, but think about it together with your knowledge and experience, and…
Sure, you can spend the weeks to months of expensive and time consuming work it takes to get a fuzzy, half accurate and biased picture of what your users workflows look like through user interviews and surveys. Or you can look at the analytics, which tell you everything you need to know immediately, always up to date, with perfect precision. Sometimes HN drives me crazy. From this thread you’d think telemetry is scre…
The problem is that, without the context of actually talking to and observing users in the real world, software teams have repeatedly misinterpreted telemetry. Even the description of how they use telemetry to decide which features need investment or improvement shows this. In the face of huge data with no context, they make bad assumptions rather than talking to actual users. Over and over again.
Re: GitHub CLI now collects pseudoanonymous telemetry
#326Earlier quoted context omitted.
> Logging every page visited is not a technical requirement of serving the requested resource. How will you know which page is having problems being served or is having performance problems?
You won’t, but that’s not what was asked. Logging the requested resource is not a technical requirement of serving that resource.
Depends how you define “technical requirement” but I’d say 404 for example is an indication of a failure to serve a given resource. If you don’t have logging you won’t know unless someone complains which means you’ll only catch the most visible issues. Same goes for performance - everywhere I’ve ever worked serving a resource was tightly coupled to “how fast can the user retrieve that resource”.
Re: GitHub CLI now collects pseudoanonymous telemetry
#327Earlier quoted context omitted.
The logical conclusion is you’re asking for no local products and everything to run server side. It’s kind of a ridiculous position that doesn’t change the spying being done other than it’s on the other side of a browser.
I accounted for this in my post. Obviously if you’re making requests just so you can spy, that’s spying.
No you didn’t. If I build you a web video editor, is that because I want to spy on you or because I want to make deployment easier and reduce install friction?
You’re making a distinction that puts you in the privileged judge position of evaluating if a service is making requests just so you can spy vs what the app author’s might believe is a critical design feature in how they want the product to operate.
Re: GitHub CLI now collects pseudoanonymous telemetry
#328Earlier quoted context omitted.
Sure, you can spend the weeks to months of expensive and time consuming work it takes to get a fuzzy, half accurate and biased picture of what your users workflows look like through user interviews and surveys. Or you can look at the analytics, which tell you everything you need to know immediately, always up to date, with perfect precision. Sometimes HN drives me crazy. From this thread you’d think telemetry is scre…
Many products would be much better if they listened to what people are saying on public forums instead of using telemetry. For example, Google Maps has a longstanding bug where it auto-translates all reviews even if they are in a language you speak. If Google cared about user feedback, they could’ve easily fixed it, but no amount of telemetry will tell them this.
The reality is that most product leaders only care about the feedback that has visible consequences. If users aren't performing some action like quitting the app that shows in the telemetry, then they aren't going to pay attention.
They'd probably call the issue you see a "craft" issue. Some PM is likely raising it. What happens is that leaders in big companies want perspectives based on data. You can go in with issues like yours but if you don't have clear data that shows significant numbers of users leaving, or users piling in, then you might as well not show up. People care about craft primarily will really struggle in these large organizations. That's not a good thing but how it is.
In large organizations, you'll see a lot of A/B testing or experimentation. Some of the worst decisions from a craft perspective are ones where they only look for "did this cause some kind of negative impact on numbers?" situation. If your feature is neutral (on abandons, uninstalls, or whatever negative outcome), then it can get shipped which overrides any qualitative question around "should we ship this in this state?". Doesn't matter too much according to these folks because it's not making things worse (in terms of numbers.)
There is probably more to explore in modern "product management" that's at the root of many of these problems. HN tends to focus on engineering but within large companies there is now a bifurcation and development of a field that forgets lots of PM was already invented.
Re: GitHub CLI now collects pseudoanonymous telemetry
#329Earlier quoted context omitted.
Yes, probably because someone else said "If you dont have analytics you are flying blind" which I initially replied to, then when you replied to my reply, I took that as agreeing with parent, which isn't necessarily true. > What I’m saying is that only relying on user interviews without supplementing them I also took your "spend the weeks to months of expensive and time consuming work [...] Or you can look at the ana…
Maybe it just comes down to how you interpret "flying blind", because I do tend to agree with that statement. Telemetry is one half of the puzzle, user interviews are the other. Without either I would argue you are flying blind; I think you agree here though.
In enterprise, you have little chance of getting the real story from end users in many cases. IT will also tell you that things are used one way, only for analytics to tell you it's the opposite. If you spend some of your UX research budget to deep dive on the area you can then finally get to the bottom of it.
I think the root of the complaints here is prioritization. The things they care about are prioritized. Qualitative feedback is likely already telling PMs that something is wrong and really should be fixed, but other feedback has more data supporting it.
Re: GitHub CLI now collects pseudoanonymous telemetry
#330Earlier quoted context omitted.
> In those environments, the telemetry being on by default means your CI fails or your Bastion host can't reach GitHub at all. i'd be surprised if the inability to submit telemetry is a hard error that crashes the program
There are definitely slightly annoying variants of this of "ah the program does its job in 200ms but takes 5s to shutdown timing out trying to send telemetry data". Especially annoying on CLI programs. I have been unpleasantly surprised by several programs outright crashing when not being able to send telemetry data consistently. Though this has usually been when the connection is a bit odd and it is able to send thr…
ran into this flavor once with a different tool, not gh. our deploy job was consistently about 8s longer than it should've been, turned out a fire-and-forget telemetry POST wasn't actually fire-and-forget when the endpoint got slow. NO_PROXY plus blackholing the host fixed it, but probably the kind of thing you shouldn't have to find via flame graph.