Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Self-host analytics for better privacy and accuracy

blog.filippo.io

Post reply on HN
Print view View original

81–90 of 93 posts

Re: Self-host analytics for better privacy and accuracy

#81

Post by MustardTiger »

kentonv wrote: post #60
MustardTiger wrote: post #20

Earlier quoted context omitted.

>Sandstorm doesn't support ARM currently because Sandstorm apps run native Linux binaries, and every app would have to be compiled for each architecture. That's true of any linux distro providing binary packages. They all support arm anyways, it is trivially simple to compile packages. Even small projects like openbsd compile tens of thousands of packages for a dozen arches.

Yes but distros accomplish that by being highly opinionated on the build process you use to build packages whereas Sandstorm tries to be unopinionated on this point. Sandstorm will support ARM someday but it's going to require a large investment in tooling in order to be painless for developers.

>Yes but distros accomplish that by being highly opinionated on the build process you use to build packages whereas Sandstorm tries to be unopinionated on this point.

How's that exactly? I looked it over and can't see any difference at all.

Re: Self-host analytics for better privacy and accuracy

#82

Post by Scirra_Tom »

pixelmonkey wrote: post #80
lewisl9029 wrote: post #65

Earlier quoted context omitted.

While I find your stance on privacy very refreshing for an analytics company, hiding your pricing info behind a sales rep is a huge turnoff for me. If you feel your pricing is reasonable for the service that you provide, I really don't see why you can't just display it proudly on your site. If SpaceX can afford to not hide their pricing behind a sales rep, so can you: http://www.spacex.com/about/capabilities

Whether to display pricing on the website is something we debated in the past, and continue to debate. (Your comment may wake up the debate for me.) Pricing for analytics services (in the marketplace) is all over the map. Google picked $150K/year as the price for GA Premium because that's the low end of an Adobe Analytics contract, who is the market leader. We're typically cheaper than existing Adobe/GA contracts. No…

I avoid all services with prices behind sales rep when possible. I always feel I'm not an astute negotiator, the sales rep will be ergo he will see me as a mug and take me for all he can get. If I do buy, even if I'm happy I'm always left wondering if I'm paying 2x as much as other customers.

Re: Self-host analytics for better privacy and accuracy

#83

Post by herbst »

_jomo wrote: post #19

Another very simple alternative is goaccess [0] which is purely server-side. It gathers quite a lot of information by parsing the server log. It doesn't do all the JavaScript stuff to track every single click, but it gives you stats on how many users visit your site, which parts, when, which sites or which domains they're coming from and also how much bandwidth they use. It also shows which status codes are coming fr…

GoAccess is really one of the greatest tools of these kind. I use it with GA on all my servers.

Re: Self-host analytics for better privacy and accuracy

#84

Post by mattab »

lucb1e wrote: post #69
smt88 wrote: post #67

Earlier quoted context omitted.

Piwik started in 2007. It was inexcusable to use md5 then. The problem is that it was ever md5, not that it is still md5. That really doesn't inspire confidence that they know what they're doing. md5 is a canary in the coal mine for me, and I definitely won't use Piwik now.

To be fair, they aren't an encryption service or something. They INSERT INTO visitor_count. So far as "inspiring confidence" needs to go for such a thing, I wouldn't write them off just yet.

It looks like it's scheduled to be fixed in Piwik 3.0.0: https://github.com/piwik/piwik/issues/5728

Re: Self-host analytics for better privacy and accuracy

#85

Post by mattab »

viana007 wrote: post #10

The biggest problem with Piwik is that is not scalable and the cost ($) with servers to store analytics data. I saw many cases that Piwik not supported the big volume of data.

Piwik is scalable up to at least 1 billion actions per month. Piwik is scalable! But it is not cheap, as one needs powerful database servers with a lot of RAM and fast SSD disks. It can be costly, but Piwik scales!

Re: Self-host analytics for better privacy and accuracy

#86

Post by mattab »

SwellJoe wrote: post #21
ljoshua wrote: post #7

Earlier quoted context omitted.

Can you elaborate a bit more?

I'd assume the prior poster is suggesting that running a third party tool on your primary web server(s) would increase the surface area for attacks by some (possibly not small) amount. e.g. if piwik is compromised, an attacker would have some sort of user access on your web server(s), which is generally a bad thing. I suppose some people also use the same database user for all web applications, which would potentiall…

See also some of the official Piwik security tips: https://piwik.org/docs/how-to-secure-piwik/

Re: Self-host analytics for better privacy and accuracy

#87

Post by mattab »

blacksmith_tb wrote: post #44
__jal wrote: post #37

I've done my own analytics since before there were hosted services. Still use an old-school, proprietary tool called Sawmill[1]. One very nice aspect (out of many) about it is that it handles hundreds of other log formats out of the box, so it can report sensibly on switches, email, firewall logs... just about anything with very little effort. No connection, other than being a long-time happy customer. [1] http://saw…

I haven't used Sawmill in ages, but it wasn't bad, once upon a time. There's also GoAccess, which is a snazzy ncurses logfile analyzer: https://www.goaccess.io/

Piwik also handle dozens of Log Formats, check it out: http://piwik.org/log-analytics/

Github project: https://github.com/piwik/piwik-log-analytics

Re: Self-host analytics for better privacy and accuracy

#88

Post by mattab »

rhizome wrote: post #39
shermozle wrote: post #31

Earlier quoted context omitted.

MySQL

I see. What is the upper bound of "big volume of data" possible under MySQL + Piwik?

I've worked on Piwik servers tracking & processing reports for up to 800 million pageview per month on hundreds of medium and larger websites.

Re: Self-host analytics for better privacy and accuracy

#89

Post by karussell »

lucb1e wrote: post #69
smt88 wrote: post #67

Earlier quoted context omitted.

Piwik started in 2007. It was inexcusable to use md5 then. The problem is that it was ever md5, not that it is still md5. That really doesn't inspire confidence that they know what they're doing. md5 is a canary in the coal mine for me, and I definitely won't use Piwik now.

To be fair, they aren't an encryption service or something. They INSERT INTO visitor_count. So far as "inspiring confidence" needs to go for such a thing, I wouldn't write them off just yet.

I love piwik but this is no excuse, sorry :) Privacy also means data protection!

Re: Self-host analytics for better privacy and accuracy

#90

Post by karussell »

Ileca wrote: post #45

"Self-hosting analytics for better privacy and accuracy." Then why using google fonts which is listed by Disconnect.me (used by Firefox) as a tracking domain? Isn't that paradoxical?

Small side story: I had a 'fun' night where I 'just' wanted to make a self-hosted wordpress also perfect regarding privacy. This was not easy! First I had to install a plugin which does not use google fonts for a certain theme, then I need to disable the default avatar service using another plugin and customization, next I needed to convert youtube and twitter embeds into images with links and so on. It took me several hours.

At that time I thought I was the only one with this requirements ... also a bit strange for an open source project to have such defaults IMO.

Post reply on HN