Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I replaced Google Analytics with simple log-based analytics

benhoyt.com

Post reply on HN
Print view View original

81–90 of 143 posts

Re: Show HN: I replaced Google Analytics with simple log-based analytics

#82

Post by redm »

This looks like a solution in need of a problem. Previously I had tried some similar things at scale, mainly because I wanted upsampled reports and GA charges $100,000 for premium services. What I found is that raw logs are not reliably accurate. The volume of traffic in certain countries was accurate but not reliably “real” users even after accounting for known bots, search engines etc. GA has a way of accounting for these and giving you a better overall picture. The second thing is that GA has improved its service so you get upsampled reports now, even at scale for tier 1 reports. At low volume, there upsampled already.

I’m not sure why anyone would want to waste time with this.

Re: Show HN: I replaced Google Analytics with simple log-based analytics

#83

Post by justkez »

johnchristopher wrote: post #55
speedplane wrote: post #38

Earlier quoted context omitted.

The real power of Google Analytics is not in the tracking code, it's with the front-end interface that any non-technical product manager or marketer can use. Tracking users is easy. Allowing non-technical folks with easy-to-use analysis tools is much harder.

The ever changing google analytics dashboard is not that easy to use for non technical folks. And then there's the whole "data analysis" thing.

The (lack of) speed and complexity of GA astounds me every time I go in (use it over a portfolio of businesses). Using it on a 100Mb connection is still like pulling teeth.

I've researched building out a desktop app that pulls GA data over the API in the background so you can get key stats out much quicker, but it's quite an investment of time to be beholden to Google's platform.

Now doing some dogfooding on a web analytics service I've been evolving that tries to answer the "why" of change in traffic/behaviour over time ("traffic's up today....not sure why?"). Google do this with their GA mobile app ("Insights") but what and when they show you don't seem to be too predictable.

Re: Show HN: I replaced Google Analytics with simple log-based analytics

#84

Post by RA_Fisher »

redm wrote: post #82

This looks like a solution in need of a problem. Previously I had tried some similar things at scale, mainly because I wanted upsampled reports and GA charges $100,000 for premium services. What I found is that raw logs are not reliably accurate. The volume of traffic in certain countries was accurate but not reliably “real” users even after accounting for known bots, search engines etc. GA has a way of accounting fo…

I prefer the raw logs. One reason is that Google performs "statistical improvement" like "sessonizing." This act destroys the value of point process data. It wouldn't be found in a stats textbook because the operation destroys valuable information. Also, the concept of Visitors and Users that GA uses isn't transparent to me.

Another fun fact. Since the tracking happens on the client side, there's potentially a ton of truncated data that GA simply misses. Backend server instruments don't suffer the same way.

Re: Show HN: I replaced Google Analytics with simple log-based analytics

#86

Post by judge2020 »

xiaq wrote: post #77
AdriaanvRossum wrote: post #72

Love what you are doing here. A few points: 1. I would rename the pixel.png to something like image.png. Never call your script anything like tracking, analytics, or pixel when you don't want to be blocked by ad blockers. We use hello.js and hello.gif. [1] > Log file parsing is an old-skool but effective way of measuring the traffic to your site. 2. By using a pixel image you can bypass caching. When using server log…

> Never call your script anything like tracking, analytics, or pixel when you don't want to be blocked by ad blockers. But this is a tracking script, no? If I were you I'd keep the name that way so that if people don't want to be tracked, they don't.

The author said it himself, he could have just put Cloudfront in front of Pages and parsed the access logs there. The referrer and user-agent, which are sent by the browser, are in the request headers. There would be no need for JS or an image if the website owner just changed their setup.

How do you deal with this sort of tracking?

Re: Show HN: I replaced Google Analytics with simple log-based analytics

#87

Post by hashedhashed »

harianus wrote: post #78
xiaq wrote: post #77

Earlier quoted context omitted.

> Never call your script anything like tracking, analytics, or pixel when you don't want to be blocked by ad blockers. But this is a tracking script, no? If I were you I'd keep the name that way so that if people don't want to be tracked, they don't.

If your business is to track people, yes. If it's to gather statistics without tracking people (like page views), I think it perfectly fine to bypass ad blockers. We even have a dedicated feature for bypassing ad blockers [1] because we think page views are not privacy invasive. We drop IP addresses from every request so there is no personal data in our database or logs. If you really want to block you can enable the…

Yeah, no. You are wrong and the fact that you provide services to go around adblockers makes you evil as fuck.

Thanks for letting me know to block hello.js and hello.gif.

Re: Show HN: I replaced Google Analytics with simple log-based analytics

#88

Post by matz1 »

xiaq wrote: post #77
AdriaanvRossum wrote: post #72

Love what you are doing here. A few points: 1. I would rename the pixel.png to something like image.png. Never call your script anything like tracking, analytics, or pixel when you don't want to be blocked by ad blockers. We use hello.js and hello.gif. [1] > Log file parsing is an old-skool but effective way of measuring the traffic to your site. 2. By using a pixel image you can bypass caching. When using server log…

> Never call your script anything like tracking, analytics, or pixel when you don't want to be blocked by ad blockers. But this is a tracking script, no? If I were you I'd keep the name that way so that if people don't want to be tracked, they don't.

The goal is to track. Allowing people to easily disable defeat the purpose.

Re: Show HN: I replaced Google Analytics with simple log-based analytics

#89

Post by finchisko »

I was thinking about switching to log based user tracking some time ago. Not because of big brother issues, but rather my intention was to remove the cookie banner non sense required by EU. No cookies, no banner required, right? I mean there are sure some downsides, but in current stage of our analytics, logs should hold enough information for analytics we need.
Post reply on HN