Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL
1–10 of 35 posts
Re: Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL
#2Hmm, it ocured to me that you can probably get a nice list of robot user-agents by querying all UAs that accessed the robots.txt file. I don't think normal browsers touch that file.
Also a thing to do on the cheap, if you want more usable logs is to do JSON logging[1] (one object per line). This is trivial to import into PostgreSQL and also trivial to query via tools like jq, as is.
[1] Example: https://stackoverflow.com/questions/25049667/how-to-generate...
Re: Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL
#3Interesting, there's quite a big number of people running ad blockers!
"Both Google Analytics and Goatcounter agreed that I got ~13k unique visitors across the couple days where it spiked. GoAccess and my own custom Athena queries agreed that it was more like ~33k unique visitors, giving me a rough ratio of 2.5x more visitors than reported by analytics, and meaning that about 60% of my readers are using an adblocker."
Re: Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL
#4I have always used GoAccess on my blog (https://2byt.es) which gets very modest traffic because I don't post much and don't advertise outside of my few twitter followers. Privacy has always been a core principle of mine.
I've found that over time, crawlers drown out the numbers of actual visitors but I find GoAccess hard to use to get any meaningful data from when interesting things do happen.
Can anyone suggest a way I can do something similar to this without relying on a service I don't host (and without having to write parsers into a SQL or similar DB by hand)?
Re: Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL
#5I've found the same issue. A lot of traffic will get blocked if you use a simple JavaScript integration. The solution is (obviously) to track from the backend and provide a simple dashboard for it. I've started building a library [0] written in Go, which I could integrate into my website and until the end of last year, it became a product (in beta right now) called Pirsch [1]. We offer a JS integration to onboard customers more easily, but one of the main reasons we build it is, that you can use it from your backend through our API [2]. We plan to add more SDKs and plugins (Wordpress, ...) to make the integration easier, but it should be fairly simple already.
I would love to hear feedback, as we plan to fully release it soon :)
[0] https://github.com/pirsch-analytics/pirsch
[2] https://docs.pirsch.io/get-started/backend-integration/
[Edit]
I forgot to mention my website, which I initially created Pirsch for. The article I wrote about the issue and my solution is here: https://marvinblum.de/blog/server-side-tracking-without-cook...
Re: Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL
#6We use goaccess against a pretty busy centralized log server and has worked really well for years. We don't have to worry about JS and that's always a plus. I personally like how it follows the unix philosophy.
Re: Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL
#7I have always used GoAccess on my blog ( https://2byt.es ) which gets very modest traffic because I don't post much and don't advertise outside of my few twitter followers. Privacy has always been a core principle of mine. I've found that over time, crawlers drown out the numbers of actual visitors but I find GoAccess hard to use to get any meaningful data from when interesting things do happen. Can anyone suggest a…
https://pirsch.io/ I'm one of the founders :) Check my post above (or below, who knows on hn?) for more details, or ask me.
[Edit] Sorry, I did read "which I don't host" instead of the other way around. You can check out the open-source core library, that might work for you if you put in some work.
Re: Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL
#8I have always used GoAccess on my blog ( https://2byt.es ) which gets very modest traffic because I don't post much and don't advertise outside of my few twitter followers. Privacy has always been a core principle of mine. I've found that over time, crawlers drown out the numbers of actual visitors but I find GoAccess hard to use to get any meaningful data from when interesting things do happen. Can anyone suggest a…
I'm waiting as well for the ability to use filters directly from goaccess. Hope they get to it soon!
https://github.com/allinurl/goaccess/issues/117
Re: Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL
#9I've found the same issue. A lot of traffic will get blocked if you use a simple JavaScript integration. The solution is (obviously) to track from the backend and provide a simple dashboard for it. I've started building a library [0] written in Go, which I could integrate into my website and until the end of last year, it became a product (in beta right now) called Pirsch [1]. We offer a JS integration to onboard cus…
Is it possible to integrate pirsch into a heroku deployment?
Re: Minimally Invasive (and More Accurate) Analytics: GoAccess and Athena/SQL
#10I've found the same issue. A lot of traffic will get blocked if you use a simple JavaScript integration. The solution is (obviously) to track from the backend and provide a simple dashboard for it. I've started building a library [0] written in Go, which I could integrate into my website and until the end of last year, it became a product (in beta right now) called Pirsch [1]. We offer a JS integration to onboard cus…
Is it possible to integrate pirsch into a heroku deployment?
I haven't worked with heroku yet, but if you can make an API request, yes. You can read about how that works here: https://docs.pirsch.io/api-sdks/api/