The biggest problem with Piwik is that is not scalable and the cost ($) with servers to store analytics data. I saw many cases that Piwik not supported the big volume of data.
Why isn't it scalable?
MySQL
Self-host analytics for better privacy and accuracy
31–40 of 93 posts
Re: Self-host analytics for better privacy and accuracy
#32There are many, many self-hosting analytics tools, from your own big data pipes to tools like the aforementioned piwik, as well as Open Web Analytics. I like Snowplow ( http://snowplowanalytics.com/ ), but it's currently hard-coded to AWS. Hosting your own analytics data can be great, but there are lots of ways to get better accuracy and control over your data without having to host everything. Still, if you can, it'…
Snowplow is only loosely hard coded to AWS. I'm using it and breaking it free is only a few hundred lines of code. For example, rerouting Snowplow's Kinesis collector into Kafka is 114 lines, and that includes logging, metrics, etc - I basically just had to extend the AbstractSink object in their scala collector. Reading from Kafka is another couple of hundred lines, similarly writing to files.
I hope you're going to push your ~200 back to the project! Would love to see it free of the AWS dependency.
Re: Self-host analytics for better privacy and accuracy
#33For those of us that prefer not to be tracked by self-hosted Sandstorm apps, 2 new ublock rules (they're very rough, but I am no ublock wizard): ||sandcats.io/embed.js$script view.gif?page=$image Note that you would have to change the filters if the scripts and tracking pixel are renamed of course but this should catch a majority of the push button installs.
Why do you object to self-hosted analytics? I understand blocking centralized trackers (I do so myself), but self-hosted doesn't seem problematic in the same way GA being present on half the pages on the Internet is.
It also strikes me as an unwinnable battle for all but the largest sites.
Re: Self-host analytics for better privacy and accuracy
#34Re: Self-host analytics for better privacy and accuracy
#35Re: Self-host analytics for better privacy and accuracy
#36For those of us that prefer not to be tracked by self-hosted Sandstorm apps, 2 new ublock rules (they're very rough, but I am no ublock wizard): ||sandcats.io/embed.js$script view.gif?page=$image Note that you would have to change the filters if the scripts and tracking pixel are renamed of course but this should catch a majority of the push button installs.
If you visit a web site the operators of that site will know that someone at your IP address visited the site, which pages you viewed, how long you lingered, which buttons you clicked, etc. If you want to prevent that your only option is to browse via things like proxies or Tor.
Re: Self-host analytics for better privacy and accuracy
#37I've done my own analytics since before there were hosted services.
Still use an old-school, proprietary tool called Sawmill[1]. One very nice aspect (out of many) about it is that it handles hundreds of other log formats out of the box, so it can report sensibly on switches, email, firewall logs... just about anything with very little effort.
No connection, other than being a long-time happy customer.
Re: Self-host analytics for better privacy and accuracy
#38Earlier quoted context omitted.
MySQL
Put Rabbit or Kafka in front of MySQL then?
They have a Redis-based solution to sit in front of mySQL (seems to be a "Pro" feature though):
Re: Self-host analytics for better privacy and accuracy
#39Re: Self-host analytics for better privacy and accuracy
#40A middle of the road approach is to use an analytics package like https://truestats.com that allows you to download raw session info at any time.