201–210 of 235 posts
Slightly off-topic: Does anyone have recommendations for self-hosted open source analytics that can handle a large volume site (think 500.000.000 impressions per month)? I can't imagine systems with MySQL/PostgreSQL as database can handle this.
out of curiosity, why so?
Earlier quoted context omitted.
I am not a lawyer so I cannot say for sure what constitutes PII and what breaches GDPR. I am using the same techniques as Fathom Analytics, Plausible.io and other products. Everything is hashed into a unique session id and none of the actual data like user agent or IP address is actually stored. It is the same data that is found in server log files. In the strictest interpretation of GDPR, I don't think any analytics…
An IP address is considered personally identifiable information in at least Germany. If you're storing that you'll already have to think about the GDPR. This is just another misguided attempt to adhere to the letter of the law while going against its spirit. Is is misguided because it's based on a wrong understand of what the letter of the law actually is. You see this a lot with adtech and analytics companies who tr…
It's easy to say this and hard to draw a line between PII and what I can store without consent. "yesterday I sold 5 products on my website" is not PII (I hope). If I store the timestamps for each purchase I'm already in the grey area. One could combine the timestamps with other data to identify my customers.
Slightly off-topic: Does anyone have recommendations for self-hosted open source analytics that can handle a large volume site (think 500.000.000 impressions per month)? I can't imagine systems with MySQL/PostgreSQL as database can handle this.
As for question - I saw a lot of great reviews on ClickHouse DB
Slightly off-topic: Does anyone have recommendations for self-hosted open source analytics that can handle a large volume site (think 500.000.000 impressions per month)? I can't imagine systems with MySQL/PostgreSQL as database can handle this.
500 000 000 requests per month is just about 200 request per second. Why there should be any problem for any DB? As for question - I saw a lot of great reviews on ClickHouse DB
Not if you assume that some hours will have more web traffic than others.
Earlier quoted context omitted.
Would randomly generating the session key instead of hashing client IP and other properties satisfy GDPR’s requirement of no PII? The definition in GDPR Art. 4 reads: [1] > ‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such…
As long as you can connect the id to one single client / user, it is PII. It does not matter, where this id comes from, a random hash, an encrypted IP adress. If it's unique, it's PII. If you only save it on the server, not on the client side, it's not PII. But then it's almost useless for analytics. Because next time the user comes around, you create another hash and therefore another user.
Ok, Ill just reuse each generated ID twice and I'm safe. The data gets only a little blurry.
Earlier quoted context omitted.
This is an awesome project. Is Postgres good enough for high traffic or would it be better to switch to Redis?
Tho very rare, why would you use a database that is at risk to be erased completely, and with a limited set of queries as the main DB? Honest question.
Earlier quoted context omitted.
Tho very rare, why would you use a database that is at risk to be erased completely, and with a limited set of queries as the main DB? Honest question.
My concern was the performance of Postgres when it's receiving thousands of writes per second. I assumed that such a task would be more suited for Redis, then the data could be filtered and sent to Postgres for longer storage (or some storage solution such as S3).
I think a time series database is best suited for this kind of project (timescale build on top of postgres, influxdb, ...)
Tackling the privacy focus for GA is great, but they're a good deal of products out there that already fill that niche, not to mention the requirements of the privacy crowd usually being a venture into itself.
If you wanted to make it relatively competitive for marketing, the simplest addition would be adding labelling via regex for referrers.
i.e. - Some users want to be able to group Baidu, Google, DuckDuckGo, into a single bucket for comparison. Some users want to break them down into common market segments by country. "https://www.baidu.com/link?url=FyYbCZqj65Vc7A4XeSNrOcQCS2qFX...
is from your live demo referrers, and makes it difficult to actually assess the amount of traffic from Baidu. Using a regex label means that users can break down traffic from Paid/Organic marketing fairly quickly, and start to build up dashboards they can use.
If you ever extended it to allow multiple labels for each hit, could re-run the regex over past data, and could build reports off it, you'd easily have a benefit over GA that would start to wean the marketing crowd off it.
Looks neat! will explore. Also, I did research on alternatives to GA few days back, might be helpful of someone: https://github.com/Open-Web-Analytics/Open-Web-Analytics https://matomo.org/ https://github.com/matomo-org/matomo https://github.com/usefathom/fathom https://www.goatcounter.com/ https://plausible.io/ https://github.com/PostHog/posthog https://www.usertrack.net/
Hi everyone! Author of Umami here. I totally did not expect this response so it looks like you all hugged my little server to death. The demo should be back up now. A little background. This is a side project I started 30 days ago because I was tired of how slow and complicated Google Analytics was. I just wanted something really simple and fast that I could browse quickly without diving through layers of menus. So I…