Live data from Hacker News

Show HN: I replaced Google Analytics with simple log-based analytics

benhoyt.com

131–140 of 143 posts

Re: Show HN: I replaced Google Analytics with simple log-based analytics

#131
post #125

Earlier quoted context omitted.

How do you know what businesses have cameras to count visitors before going into the business?

Noone puts up cameras to count the visitors. They put them to monitor their activity.

Actually some people do, and a lot of people use their existing surveillance cameras for people-counting. see for example https://www.axis.com/en-us/products/axis-people-counter

Re: Show HN: I replaced Google Analytics with simple log-based analytics

#132

Earlier quoted context omitted.

The real power of Google Analytics is not in the tracking code, it's with the front-end interface that any non-technical product manager or marketer can use. Tracking users is easy. Allowing non-technical folks with easy-to-use analysis tools is much harder.

The ever changing google analytics dashboard is not that easy to use for non technical folks. And then there's the whole "data analysis" thing.

> The ever changing google analytics dashboard is not that easy to use for non technical folks.

True, but easier to learn than custom programming and visualizations. A smart English major can still figure it out on their own. Giving product managers the tools to do in-depth analysis is a huge plus. Otherwise, they just ask devs to run reports all the time, burdening devs and slowing down analysis.

Re: Show HN: I replaced Google Analytics with simple log-based analytics

#133

I also would suggest looking into Matomo ( https://matomo.org/ ) if your want an open source analytic service to replace Google Analytics.

FYI, Matomo used to be called Piwik, for anyone more familiar with their old name.

https://matomo.org/blog/2018/01/piwik-is-now-matomo/

Re: Show HN: I replaced Google Analytics with simple log-based analytics

#134
Very nice!

On several projects, we've had success with a custom tracker that records IP, URL, referrer, display resolution, OS, and user agent to a local db.

To filter out bot traffic, we used Crawler-Detect [1].

The whole thing is just a few lines of PHP and JS, doesn't even require a tracking pixel (we grab most of the data from the user session).

A cron job moves entries older than x from the production db to an archive db.

[1] https://github.com/JayBizzle/Crawler-Detect

Re: Show HN: I replaced Google Analytics with simple log-based analytics

#135
post #2

If you’re not a fan of big brother you probably should be logging to bare metal instead of Amazon S3.

https://aws.amazon.com/compliance/data-privacy-faq/ "We do not access or use your content for any purpose without your consent. We never use your content or derive information from it for marketing or advertising"

[deleted]

Re: Show HN: I replaced Google Analytics with simple log-based analytics

#136
post #80

Earlier quoted context omitted.

If your business is to track people, yes. If it's to gather statistics without tracking people (like page views), I think it perfectly fine to bypass ad blockers. We even have a dedicated feature for bypassing ad blockers [1] because we think page views are not privacy invasive. We drop IP addresses from every request so there is no personal data in our database or logs. If you really want to block you can enable the…

The privacy game is about power, not about who is doing what right now. People shun Google's data collection because of what Google can do with the data, not what it has done or is doing; it only takes a single case of data misuse to reveal the power dynamics even if nothing has happened to them personally. You don't have to play the privacy game -- there is a lot of space between really respecting user's privacy and…

https://twitter.com/randfish/status/1126892446925848578?s=21

Re: Show HN: I replaced Google Analytics with simple log-based analytics

#138

Earlier quoted context omitted.

I don't think what you're doing is the end of the world, but answer this is extremely patronizing. You've basically said you know some users block your stuff, but you think they probably shouldn't really want to block your stuff, so you devise an end run around them. Then, finally, you conclude that if they really want to block your stuff, they should do something that you yourself admit won't block your stuff, but t…

I'm fine with people being patronizing towards whiny, entitled users who think they have a right to use any website they want on their terms. If you don't like someone running log analysis on their traffic then don't go to their website, nobody forced you to.

This example of tracking is tiny and harmless. But there's a wide spectrum of tracking behavior from sites.

> whiny, entitled users who think they have a right to use any website they want on their terms

Users do have a right to use any website they want to on their own terms. If I make an HTTP GET to your site, it's up to you to decide what HTML to return. Once you do, it's up to me whether to request the images, scripts, etc, and whether to read the sidebar, etc.

It's up to you to decide whether to show me any content of substance without first collecting payment. I can't demand that you publish content for free. I'm not entitled to that.

But it's up to me to decide what I consume. You can't demand that I view ads, send back tracking cookies, etc. You're not entitled to that.

If you don't like site visitors refusing to be tracked, then don't let them view your website. Nobody forced you to.

Re: Show HN: I replaced Google Analytics with simple log-based analytics

#139

i’ve heard snowplow is good for this

Totally agree! I'm running a minimalistic version of snowplow's collection/etl infra for under $1 per month, and it works great:

https://bostata.com/client-side-instrumentation-for-under-on...

Re: Show HN: I replaced Google Analytics with simple log-based analytics

#140
(data engineer here)

Nice post! It's always fun reading about people being creative and challenging the analytics status quo (aka GA). Besides the joy of doing it yourself, you've accomplished a couple other things worth mentioning:

1. You'll never be sampled. GA samples historical data pretty heavily, and you have to pay for 360 to retain unsampled event data (at a tune of $160k+ per year).

2. You have full access to all generated data.

I'd highly recommend using Snowplow's javascript tracker (https://github.com/snowplow/snowplow-javascript-tracker) in a very similar manner to what you've outlined here. You'll get a ton of extra functionality out of the box, which would add yet another level of insight. With snowplow, you get the following for free:

1. Sessionization, which is consistent with google analytics' definition - effectively a 30 minute window of activity.

2. User identification - the tracker drops a persistent cookie (just like GA), so you can see returning visitors.

3. Tools for splitting requests

4. A variety of event types, out of the box: https://github.com/snowplow/snowplow/wiki/2-Specific-event-t...

5. Ability to respect Do Not Track

6. Time on page, browser width/height, etc

7. Ability to make your event tracking 100% first-party

(Disclaimer: I don't work for them, but I've seen the system work very well a number of times.)

I'm running a similar setup on my blog, and it costs well under $1 per month: https://bostata.com/client-side-instrumentation-for-under-on.... I'm doing the same exact thing with Cloudfront log forwarding and have several lambdas that process the files in S3. From there, I visualize traffic stats with AWS Athena (but retain a ton of flexibility, since they are all structured log files).

Post reply on HN