Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL

brandur.org

Post reply on HN
Print view View original

21–30 of 35 posts

Re: Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL

#21

Post by conradfr »

alias_neo wrote: post #4

I have always used GoAccess on my blog ( https://2byt.es ) which gets very modest traffic because I don't post much and don't advertise outside of my few twitter followers. Privacy has always been a core principle of mine. I've found that over time, crawlers drown out the numbers of actual visitors but I find GoAccess hard to use to get any meaningful data from when interesting things do happen. Can anyone suggest a…

There is the --ignore-crawlers argument, on my modest projects it seems efficient but I've not look at it too precisely.

Re: Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL

#22

Post by mrwnmonm »

herunan wrote: post #16
alias_neo wrote: post #4

I have always used GoAccess on my blog ( https://2byt.es ) which gets very modest traffic because I don't post much and don't advertise outside of my few twitter followers. Privacy has always been a core principle of mine. I've found that over time, crawlers drown out the numbers of actual visitors but I find GoAccess hard to use to get any meaningful data from when interesting things do happen. Can anyone suggest a…

I use Cloudflare Web Analytics. Since I use Cf, I thought: why not utilize their analytics? Anonymous, no cookie, no fingerprinting and no localStorage. Edit: also, no JS if you have Pro (20usd/mo).

I read that DNS analytics is not accurate too.

Re: Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL

#23

Post by mrwnmonm »

marvinblum wrote: post #7
alias_neo wrote: post #4

I have always used GoAccess on my blog ( https://2byt.es ) which gets very modest traffic because I don't post much and don't advertise outside of my few twitter followers. Privacy has always been a core principle of mine. I've found that over time, crawlers drown out the numbers of actual visitors but I find GoAccess hard to use to get any meaningful data from when interesting things do happen. Can anyone suggest a…

https://pirsch.io/ I'm one of the founders :) Check my post above (or below, who knows on hn?) for more details, or ask me. [Edit] Sorry, I did read "which I don't host" instead of the other way around. You can check out the open-source core library, that might work for you if you put in some work.

Wow, Go support, thanks.

Re: Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL

#24

Post by XCSme »

mrwnmonm wrote: post #22
herunan wrote: post #16

Earlier quoted context omitted.

I use Cloudflare Web Analytics. Since I use Cf, I thought: why not utilize their analytics? Anonymous, no cookie, no fingerprinting and no localStorage. Edit: also, no JS if you have Pro (20usd/mo).

I read that DNS analytics is not accurate too.

Semi-related question: Is any type of web analytics 100% accurate?

Re: Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL

#25

Post by mrwnmonm »

XCSme wrote: post #24
mrwnmonm wrote: post #22

Earlier quoted context omitted.

I read that DNS analytics is not accurate too.

Semi-related question: Is any type of web analytics 100% accurate?

I meant not as accurate as client tracking if ad blockers were not used. As far as I understand it.

Re: Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL

#27

Post by XCSme »

mrwnmonm wrote: post #25
XCSme wrote: post #24

Earlier quoted context omitted.

Semi-related question: Is any type of web analytics 100% accurate?

I meant not as accurate as client tracking if ad blockers were not used. As far as I understand it.

But even if ad-blockers were not used, people still disable JS, JS files fail to load, JS takes longer to load than the user stays on the site and many other potential issues.

My comment was not directly related to yours, I agree that DNS analytics are probably worse, I was just wondering if theoretically is it possible to produce high-accuracy analytics when everything can be spoofed/cached, etc.

Re: Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL

#28

Post by joshyi »

mrwnmonm wrote: post #25
XCSme wrote: post #24

Earlier quoted context omitted.

Semi-related question: Is any type of web analytics 100% accurate?

I meant not as accurate as client tracking if ad blockers were not used. As far as I understand it.

Tracking visitors at the client level deflates the actual number of visitors. On the other hand, server-side tracking provides a more accurate number with the tradeoff of not knowing for sure if the client is a human behind a browser.

Re: Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL

#29

Post by alias_neo »

conradfr wrote: post #21
alias_neo wrote: post #4

I have always used GoAccess on my blog ( https://2byt.es ) which gets very modest traffic because I don't post much and don't advertise outside of my few twitter followers. Privacy has always been a core principle of mine. I've found that over time, crawlers drown out the numbers of actual visitors but I find GoAccess hard to use to get any meaningful data from when interesting things do happen. Can anyone suggest a…

There is the --ignore-crawlers argument, on my modest projects it seems efficient but I've not look at it too precisely.

I've always had this flag in my run script but find I still have huge amounts of crawler traffic. I might need to look at that again.

Re: Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL

#30

Post by bitsondatadev »

twotwotwo wrote: post #19

I want to second that plug for Athena for ad-hoc analysis. (If you're hosting your own stuff and at the scale where it'd be useful, there's Presto/Hive, which Athena is based on, and/or at Trino, the Presto fork maintained by some of its initial developers.) It was useful for me when tweaking spam/bot detection rules a while ago; if I could roughly describe a rule in a query, I could back-test it on old traffic and f…

One quick detail on the Trino description is that not only are some of the initial developers but all of the creators and the majority of contributors (https://github.com/prestodb/presto/graphs/contributors?from=...) and still have contributed the majority of the code in both Presto (https://github.com/prestodb/presto/graphs/contributors)/Trin... (https://github.com/trinodb/trino/graphs/contributors).

To really jump into this, take a look at https://trino.io/blog/2020/12/27/announcing-trino.html.

A few more stats and info:

Trino commits: 22,383 Presto commits: 18,582

Trino slack members: 3,603 Presto slack members: 1,575

Trino supports iceberg: https://trino.io/docs/current/connector/iceberg.html HDP3 Support: https://github.com/trinodb/trino/issues/1218

Trino has addressed a critical security vulnerability that still exists in Presto: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-1508...

Give our repo a star if you have a sec: https://github.com/trinodb/trino/blob/master/.github/star.pn...

Post reply on HN