Shameless plug: We're also working on an on-premise custom analytics platform that can be deployed either Heroku or AWS with Cloudformation. https://github.com/rakam-io/rakam
Self-host analytics for better privacy and accuracy
51–60 of 93 posts
Re: Self-host analytics for better privacy and accuracy
#52Earlier quoted context omitted.
I don't think* embedding Google Fonts allows Google much (any?) data collection beyond, presumably, that the font was requested, though it's definitely an unnecessary dependency. Given Google Fonts allows you to download all of the fonts, with license info provided, in a variety of formats, there's almost no good reason not to embed them directly in your own site. Right now, Sandstorm apps will currently work with Go…
Google Fonts does see the page you are on, in the `Referer` header. According to mitmproxy: host: fonts.googleapis.com Connection: keep-alive Proxy-Connection: keep-alive Accept: text/css,*/*;q=0.1 User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 9_3_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13E238 Safari/601.1 Accept-Language: en-us Referer: https://blog.filippo.io/self-host-analytic…
And it does appear Google just blankets this into it's general API terms of use that they can "use submitted data" in accordance with their general privacy policies. So yeah, I guess they can use it as part of their tracking. :/
Re: Self-host analytics for better privacy and accuracy
#53Piwik is a great project, but it tends not to work well for handling sites with millions of events per day. Your MySQL table starts to bust at the seams pretty quickly. For big sites, you'll want that event data in GiB's of plain raw logs that you can bulk load into tools like BigQuery or Redshift for analysis. My team has built/delivered a SaaS web content analytics platform for the past few years called Parse.ly. W…
Is their any difference in data ownership between GA and parsely?
Re: Self-host analytics for better privacy and accuracy
#54Earlier quoted context omitted.
How does snowplow solve this problem? By writing to disk?
Snowplow is implemented as a unidirectional data pipeline: tracking -> collection -> enrichment -> storage Between each step there is typically some kind of persistent queue (mostly S3/Kinesis), and data won't be lost if a downstream component is not operational. Examples: * If your event collector is unavailable, raw events will be cached in the tracker in localStorage, SQLite or similar * If your Redshift database…
There is of course the opportunity for a collector to be down, but the aim is to keep those components super simple and rely on really durable storage (Kinesis, S3) managed by someone else.
Re: Self-host analytics for better privacy and accuracy
#55Earlier quoted context omitted.
MySQL
I see. What is the upper bound of "big volume of data" possible under MySQL + Piwik?
That's a difficult thing to answer. However the more important problem is loss of data whenever your DB isn't available due to downtime, upgrade etc. It depends how important data loss is for your user case. I'm a data completist but I'm in therapy for it ;)
It's definitely worth playing with, and trivially easy to spin up. Other self-hosted options aren't anything like as simple to get up and running.
Re: Self-host analytics for better privacy and accuracy
#56Piwik is a great project, but it tends not to work well for handling sites with millions of events per day. Your MySQL table starts to bust at the seams pretty quickly. For big sites, you'll want that event data in GiB's of plain raw logs that you can bulk load into tools like BigQuery or Redshift for analysis. My team has built/delivered a SaaS web content analytics platform for the past few years called Parse.ly. W…
Is their any difference in data ownership between GA and parsely?
Data ownership in GA is a "gray area" that becomes less gray if you pay $150K/yr for "GA Premium".
Google has mixed incentives in running its free analytics service. It gets web-wide analytics data, it uses data to help it sell more AdWords to customers, and it integrates GA with other services, like their display advertising products (DFP, etc.)
From a practical standpoint, you don't "own" analytics data when a) you can't easily access it in raw form and b) the SaaS provider "leaks" your data to dilute its value to you. We address (a) and (b) directly through our products and public data privacy stance. See this blog post for our public view on analytics data privacy:
http://blog.parsely.com/post/3394/analytics-privacy-without-...
Re: Self-host analytics for better privacy and accuracy
#57Google Analytics has Measurement Protocol which allows for posting data server-side.
All you need to do is proxy the data collection and then send it to them, taking advantage of all the scalability and features they have.
https://developers.google.com/analytics/devguides/collection...
Re: Self-host analytics for better privacy and accuracy
#58Earlier quoted context omitted.
I see. What is the upper bound of "big volume of data" possible under MySQL + Piwik?
That's a difficult thing to answer. However the more important problem is loss of data whenever your DB isn't available due to downtime, upgrade etc. It depends how important data loss is for your user case. I'm a data completist but I'm in therapy for it ;) It's definitely worth playing with, and trivially easy to spin up. Other self-hosted options aren't anything like as simple to get up and running.
Well, but the point was MySQL + Piwik "does not scale" and that it's "expensive" besides, which doesn't comport with my experience and sounds like received wisdom.
Re: Self-host analytics for better privacy and accuracy
#59Earlier quoted context omitted.
Why do you object to self-hosted analytics? I understand blocking centralized trackers (I do so myself), but self-hosted doesn't seem problematic in the same way GA being present on half the pages on the Internet is. It also strikes me as an unwinnable battle for all but the largest sites.
Because OP is against all kind of tracking? And because he can...
I can't claim to speak for OP, but am also against most tracking. I would also tend to think that being against first party tracking would be an unwinnable battle. It also leaks less data than third-party tracking, since the third party can see your activity across multiple sites whereas first party can only see your activity on that site unless it's aggregated through a backend service (another poster mentioned the ability to upload server logs to GA). No matter what, they can see what you load from their site.
Getting to first party hosting of more intrusive analytics (scroll location, etc) I think rather than disallowing certain scripts/URLs to run, you have to get back to behavioral-based blocking. Doing that in an environment that you allow any JS to execute seems tough since sandboxing something that can update the page based on location can "talk" to another part that can report back to the server.
If you don't like intrusive first party analytics, just stop all JS.
Re: Self-host analytics for better privacy and accuracy
#60Earlier quoted context omitted.
Just notes on this for the curious: 1. Sandstorm doesn't support ARM currently because Sandstorm apps run native Linux binaries, and every app would have to be compiled for each architecture. 2. I honestly think you'd be running pretty crippled trying to do Sandstorm on a RasPi. It's a bit smaller scale than Sandstorm seems targeted for. Each open Sandstorm grain commonly uses 100 MB of RAM or more (on top of the RAM…
>Sandstorm doesn't support ARM currently because Sandstorm apps run native Linux binaries, and every app would have to be compiled for each architecture. That's true of any linux distro providing binary packages. They all support arm anyways, it is trivially simple to compile packages. Even small projects like openbsd compile tens of thousands of packages for a dozen arches.
Yes but distros accomplish that by being highly opinionated on the build process you use to build packages whereas Sandstorm tries to be unopinionated on this point.
Sandstorm will support ARM someday but it's going to require a large investment in tooling in order to be painless for developers.