Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Counting Ghosts: A case for abandoning web analytics

pcmaffey.com

Post reply on HN
Print view View original

91–100 of 100 posts

Re: Counting Ghosts: A case for abandoning web analytics

#91

Post by xg15 »

Yeah, I don't know. I don't like ads and tracking either, but this post just seems to argue to throw them away without any kind of replacement.

I can kinda agree with the section describing all the problems with tracking, but there is not really an alternative approach presented.

The last paragraphs through in lots of buzzwords and lofty ambitions ("holistic approach") but don't give any indication how to realize those except for "go by your feelings".

I consider myself a good developer with useful debugging skills. I often "went by my feelings" when trying to find the root cause of a bug - however, to be able to do that, I need some information about the program and usually some way to interact and test hypotheses. If I've gathered enough information like this, then I can use experience to make educated guesses about what is going wrong.

Likewise, in the "real world", we always have a minimum of observable feedback that helps us judge our own actions: How people react to what we do, their body language, etc.

This is true both for individuals and for businesses: If you run a small corner store, you'll also be able to get a rough measure of success of your store just by watching how many people make come in and how many it to the counter.

In contrast, on the internet, there is, by default, no feedback at all - if you make a website and upload it to a server somewhere, it's more like running a radio station: Talking to the void, not knowing if hapf the city is listening, or only a handful or no one at all

So in order to get any information about whether what you are doing makes any sense, you need feedback.

I don't think intrusive tracking and surveillance is the only way to get feedback though. I think very basic analytics, such as counting the page hits, referral and "user journey" throughout your site are ok - those aren't very different from counting customers in a physical store.

For more detailed information, I think there should be more approaches to gather active feedback - i.e. give users a way to leave a message, point out issues they see themselves, etc.

It used to be hard to analyse this kind of written feedback at scale, but maybe LLMs could help with that in the future.

Re: Counting Ghosts: A case for abandoning web analytics

#92

Post by exodust »

josefresco wrote: post #80
digitalengineer wrote: post #61

Earlier quoted context omitted.

GA4 is actually the simple version of GA. Lots of stuff is hidden. So it looks lol Google did exactly what you wanted )‘simplified Analytics service’). However, if you want an even simpler solution: Matamo is totally free as well.

GA4 which I've migrated about 250 clients to (that was a ton of fun /s) simplifies some things, but complicates others. I know of the alternatives but run into anther issue relating to cost: Most of my clients are not willing to pay (more) for analytics which is why GA remains the default (free) choice. Do I raise prices for everyone and include an analytics alternative? Or do I stop using GA and only give (alternati…

Don't suppose you've noticed Analytics or Google Tag Manager slower than before? I've noticed when our page loads, lingering sluggish connections related to GTM, even when there's no tags apart from analytics. Prior to GA4 I never noticed this.

Also Google seems desperate for Analytics users to send their user/customer data to Google, via "Google Signals" and User-id. Google wants this normalized. They use a murky mix of ambiguous settings and labels in Analytics to present the case for these data collection features to be activated. I recall a Google Ads account manager sent a direct email to my non-technical boss with steps for moving from UA to GA4. "You need to do these steps" it read. Step 3 was "enable Google signals". They implied it was required for GA4 when it isn't. I thought that was grubby. "Used car salesman" grubby.

Re: Counting Ghosts: A case for abandoning web analytics

#93

Post by nine_k »

exodust wrote: post #90
nine_k wrote: post #75

Earlier quoted context omitted.

At my previous job, A/B testing was widely used to check which of the small UX tweaks works better.

The problem there is many uncontrollable factors contributing to the data. If engagement is found to be better with A on the day, does that mean B will never be better? It doesn't. A/B testing is often conducted unscientifically or with insufficient sample size and timeframe. Sometimes the cards fall where they fall, and your small UX tweak wasn't involved. It's tempting to conclude A/B testing delivered a valid resu…

This is fair; valid experiments need to run for weeks, not hours, to account for cyclic variations, should not be confounded by some events (like holidays, elections, etc) that severely change relevant user activity, etc, and need to bring large enough amounts of data points for the results to be able to be statistically significant.

Re: Counting Ghosts: A case for abandoning web analytics

#94

Post by gfodor »

barnabee wrote: post #69
gfodor wrote: post #36

I don't get it - it is a nice sentiment, but the crux of the argument seems to "go with your gut." The reason web analytics has been so popular is that you can actually improve your product significantly without as much subjectivity. Before web analytics, going with your gut was the only option. It turns out, threading events through a funnel that show you lost 90% of your users because your button doesn't show up on…

> improve your product significantly without as much subjectivity It's fake objectivity that often leads to a race to the boottom. Between "engagement" as a terrible proxy for user happiness/success and the idea that it's more important and better if nobody can fail to use your app's "UX" than it is if it to enables a smaller number of people who take the time to learn to do really great things, all wrapped up in the…

I didn’t say it was objective. I said it allows you to be less subjective. I have no idea how anyone could think otherwise unless you’ve never had, for example, analytics show you your visual design or interactive flow is literally broken on specific but rare device/platform/browser/etc combos. (Just one example.)

As usual, it’s possible to be stupid and abuse tools, but what I wrote presumes you’re not doing something stupid. If you think it’s impossible to not be stupid when it comes to this tool, this is false.

Re: Counting Ghosts: A case for abandoning web analytics

#95

Post by gfodor »

exodust wrote: post #90
nine_k wrote: post #75

Earlier quoted context omitted.

At my previous job, A/B testing was widely used to check which of the small UX tweaks works better.

The problem there is many uncontrollable factors contributing to the data. If engagement is found to be better with A on the day, does that mean B will never be better? It doesn't. A/B testing is often conducted unscientifically or with insufficient sample size and timeframe. Sometimes the cards fall where they fall, and your small UX tweak wasn't involved. It's tempting to conclude A/B testing delivered a valid resu…

Well yeah you have to actually do it right.

Re: Counting Ghosts: A case for abandoning web analytics

#96

Post by gfodor »

pcmaffey wrote: post #76
gfodor wrote: post #36

I don't get it - it is a nice sentiment, but the crux of the argument seems to "go with your gut." The reason web analytics has been so popular is that you can actually improve your product significantly without as much subjectivity. Before web analytics, going with your gut was the only option. It turns out, threading events through a funnel that show you lost 90% of your users because your button doesn't show up on…

The pendulum has swung too far. We quantify everything now, chasing numbers to the point of losing our sense of quality and craftsmanship. There's a huge difference between "going with your gut" and the ability to assess, qualify, and reason about something. To understand how and why something works or doesn't. Whether from first principles or experience. Numbers provide a happy path for consensus. If the "why" of yo…

The original post proposes they be abandoned. So I’m arguing against that.

Re: Counting Ghosts: A case for abandoning web analytics

#97

Post by gfodor »

hedora wrote: post #73
gfodor wrote: post #66

Earlier quoted context omitted.

What I wrote wasn't an insistent denial, try steelmanning next time. Extrapolating the magnitude of problems from user research is hard, since you're still sampling a distribution.

If the already-released version of your product has so many problems that you have to use computational techniques to extrapolate their magnitude, then you have already lost.

You can find problems like “oops, the Android they use in India on phone model X has chrome that blocks the screen every week so they can’t get passed the login flow”

It’s about better sampling of the actual distribution. User testing has huge selection bias.

Re: Counting Ghosts: A case for abandoning web analytics

#98

Post by gfodor »

hedora wrote: post #72
gfodor wrote: post #36

I don't get it - it is a nice sentiment, but the crux of the argument seems to "go with your gut." The reason web analytics has been so popular is that you can actually improve your product significantly without as much subjectivity. Before web analytics, going with your gut was the only option. It turns out, threading events through a funnel that show you lost 90% of your users because your button doesn't show up on…

You could hire a decent QA department. Then you'd run the app on all screen sizes before each git push and notice that the button doesn't always show up, or isn't always clickable.

The point is that there is a swath of failure modes that only become apparent when you actually have instrumentation point at them first. QA can’t fish these out.

Re: Counting Ghosts: A case for abandoning web analytics

#99

Post by julianeon »

barnabee wrote: post #71
julianeon wrote: post #67

If you don't have any analytics, then how do you know your audience size? Is it 10 people per month? 1000 people? 1,000,000 people? No offense but I can't accept "it doesn't matter."

Vanity metric. The only thing you need to know is how many people paid for your product.

How many people bounced off your page without buying anything? Was it 90%? 99%? 99.99%?

You can't optimize for stuff like that, and improve conversion, without that data.

Re: Counting Ghosts: A case for abandoning web analytics

#100

Post by wackget »

gfodor wrote: post #36

I don't get it - it is a nice sentiment, but the crux of the argument seems to "go with your gut." The reason web analytics has been so popular is that you can actually improve your product significantly without as much subjectivity. Before web analytics, going with your gut was the only option. It turns out, threading events through a funnel that show you lost 90% of your users because your button doesn't show up on…

> show you lost 90% of your users because your button doesn't show up on certain screen sizes

How does analytics do that? A mobile usability scanner might pick that up, but analytics?

Post reply on HN