Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we built a GDPR-compliant website analytics platform without using cookies

usefathom.com

Post reply on HN
Print view View original

101–110 of 129 posts

Re: How we built a GDPR-compliant website analytics platform without using cookies

#101

Post by JackWritesCode »

gcbw2 wrote: post #99

This is still logging everything the GDPR says you can't without asking for consent, but you made your search convoluted (but not less efficient if you have all the pieces) to (suggest|lie?) that you need to break the hash and that's why you don't need consent. None of the information you are using on the hash wouldn't be in the search query itself! ip, user agent, path, date, etc. So there is no way to reverse the h…

You're focused mostly on Recital 26, which was only a theory of mine, outside of that we are GDPR compliant anyway. I likely shouldn't have included it since that isn't our primary ground for processing. Please see: https://usefathom.com/data/

And yes the daily hash gets stored until midnight. But what are you talking about with 'search query' containing IP, user agent etc.?

Re: How we built a GDPR-compliant website analytics platform without using cookies

#102

Post by JackWritesCode »

nybble41 wrote: post #100
JackWritesCode wrote: post #95

Earlier quoted context omitted.

I do take your point but when we get into this area, it becomes a big question of trust. Because if I gave you a hash, you would need to guess the 256 bit salt in addition to all the other possibilities. I mean, hey, here's a hash: 26246226167b9f190d3a1ce726efe07ae18bbf0480a78d19390b9aaf13f25cb0 Imagine you just got hold of it through a data breach. I'll give you $1,000 if you can get it de-hashed before midnight Chi…

It's true that you'd need to know the salt first, but the server does know the salt, and if you obtained the hash through a data breach then you probably obtained the current day's salt as well.

So if someone had unlimited access to our servers, that would be a problem. One piece to note is that page views get deleted after around 30 minutes.

So one of the reasons we posted to HN was for conversations like this. Reading what you put makes me think we need to do more when creating the PageRequestSignature and the SiteRequestSignature, because if someone had access to our server, got the hash, stopped all cron jobs from processing data, then they could work it out after a huge amount of time / computing power. But to be honest, at this point, they could also add log($_SERVER) and get the entire request body of a user, so we would have much bigger problems in that scenario.

Anyway, you've given me a few new thoughts on hardening from data breaches. Because, yes, if they know the hash & have full access to our server then it becomes easier. So we almost need to move it to the point where they won't know the salt that was used for a particular user. So we'd not recycle a single salt, we'd recycle multiple salts that are based on, perhaps, the first 2-3 digits of a users IP address combined with the last 2-3 digits.... Then it would be much harder to break without first knowing the $_SERVER dump (the user's IP etc.). Obviously this wouldn't stop a "complete control of server" attack where they could just start logging everything but it would really ruin a brute forcers day because they wouldn't know what salt to start with.

What do you think of that idea? I'm running on little sleep so be nice ;) Also thanks so much for all your feedback so far, it's so appreciated!

Re: How we built a GDPR-compliant website analytics platform without using cookies

#103

Post by tomp »

Maybe I'm missing something but (1) I don't think this is GDPR compliant, and (2) why so complicated?

Regarding (1),

> Brute forcing a 256 bit hash would cost 10^44 times the Gross World Product (GWP). [...]

> We have rendered the data anonymous to the point where we could not identify a natural person from the hash.

> It's possible that GDPR does not apply to Fathom since data is made completely anonymous. Even if GDPR did still apply, we reiterate the stance that there is legitimate business interest to understand how your website is performing.

This seems to imply a profound confusion between the difference of hashing vs. anonymity. Just because it's hashed doesn't mean it's anonymous! You don't need to "brute-force" the hash, you just need to find a user that matches your hash... which is 1 in 7 billion (or so), much more tractable. This is also the principle e.g. MD5 rainbow tables are based on...

They claim to change the hash every 24 hours, so it's equivalent to having a session cookie with 24-hour expiration (session cookies are "anonymous" by their definition, they don't have any user information and they're impossible to "brute force", they "just" enable tracking). I've no idea if 24-hour session cookies are GDPR-compliant...

Regarding (2), given that this seems (again, I might be misunderstanding) equivalent to a 24-hour session cookie, why not just do that? However, then you're ... drumroll ... giving control to the user. Why not just give control to the user, period?! For example, by storing the list of pages visited in Local Storage, and only pinging the server once for each page(view) every 24 hours?

Re: How we built a GDPR-compliant website analytics platform without using cookies

#104

Post by M2Ys4U »

JackWritesCode wrote: post #46
munchbunny wrote: post #41

Earlier quoted context omitted.

> Does this meet the requirements for a site to avoid notifying users under the GDPR? I have no idea. Not necessarily. It's a bit complex, but the fact that Fathom ingests personal data at all likely means that they must still be disclosed by whoever is using Fathom's code. On the other hand, if Fathom were able to push data ingestion into the first party's infrastructure so that only aggregated data hits Fathom's ow…

What do you think about Recital 26? For GDPR, our stance can be found here: https://usefathom.com/data/

(obligatory IANAL) Recitals aren't legally binding in and of themselves, so I'd be wary of _relying_ on them _completely_.

That said, the articles of a regulation should be read in light of the recitals (as the latter are the rationale behind the actual binding legislation).

Re: How we built a GDPR-compliant website analytics platform without using cookies

#105

Post by JackWritesCode »

tomp wrote: post #103

Maybe I'm missing something but (1) I don't think this is GDPR compliant, and (2) why so complicated? Regarding (1), > Brute forcing a 256 bit hash would cost 10^44 times the Gross World Product (GWP). [...] > We have rendered the data anonymous to the point where we could not identify a natural person from the hash. > It's possible that GDPR does not apply to Fathom since data is made completely anonymous. Even if G…

For GDPR compliance notes please see: https://usefathom.com/data/

> You don't need to "brute-force" the hash, you just need to find a user that matches your hash... which is 1 in 7 billion (or so), much more tractable. This is also the principle e.g. MD5 rainbow tables are based on...

Not quite. We use a SHA256 hash as our salt, and that changes each day, so you'd need to brute force that.

In terms of how many possible combinations there are for this salt, please see: https://stackoverflow.com/a/49520766 - you would need to brute force it and try each possible combination with every single possible IP / User Agent / Site combination to break a hash. This is why it's not theoretically impossible but it's practically impossible.

We would love to approach things in an easier way but PECR doesn't want cookies, even anonymous ones.

Now, one thing that we have uncovered thanks to someone on here is that we need to increase our resistance to data breaches. If someone had complete, unlimited access to all our data / servers, including the daily salt, then they could de-hash page views from the last 30 minutes. I have no idea how long that would take. There are 4,294,967,296 (?) possible IP addresss, and then over 3M (?) user agents, so it'd be an absurd, pointless exercise.... Anyway, we're going to be bringing in multiple salts that depend on the user IP address, meaning that, in the event of a data breach, a hacker won't know which salt has been used for a hash :) Perhaps we base the salts on the first 3 digits of an IP address? That would mean we'd have 720 possible SHA256 salts!

Re: How we built a GDPR-compliant website analytics platform without using cookies

#106

Post by JackWritesCode »

M2Ys4U wrote: post #104
JackWritesCode wrote: post #46

Earlier quoted context omitted.

What do you think about Recital 26? For GDPR, our stance can be found here: https://usefathom.com/data/

(obligatory IANAL) Recitals aren't legally binding in and of themselves, so I'd be wary of _relying_ on them _completely_. That said, the articles of a regulation should be read in light of the recitals (as the latter are the rationale behind the actual binding legislation).

Trust me, we're not relying on Recital 26, that is just my thought experiment :) See here for our GDPR compliance: https://usefathom.com/data/

Re: How we built a GDPR-compliant website analytics platform without using cookies

#108

Post by jdlshore »

JackWritesCode wrote: post #37
joekrill wrote: post #24

Earlier quoted context omitted.

There's certainly nothing wrong with that. But this sounds a little defensive. And if you're going to promote the "Community Edition"/open-source nature of the codebase, this should probably be made more clear to your users/contributors. (Maybe it is made clear and I'm just not seeing it, though - this is my first time hearing about this product).

I think the defensiveness comes when people put comments such as "I'm a little concerned by the lag in publishing this update". I've never spoken like this in my life to an OS contributor and I use open-source software every day. We're working hard to get the software OS but we were originally planning on keeping the codebases separate (since they were different languages). We only recently pivoted on this after spea…

Speaking as a paying customer of Fathom [1], thank you for your hard work. I know how hard it is to spin up a new business. I for one don't care whether your code is open source or not... I'm paying for the service and the code has no value to me. Please spend your limited time and money on building your business. I don't want the distraction of having to find another analytics platform. ;-)

[1] https://www.agilefluency.org

Re: How we built a GDPR-compliant website analytics platform without using cookies

#109

Post by hedora »

SCLeo wrote: post #73

Looking at their live demo ( https://stats.usefathom.com/#!p=1w&g=hour ), I can see a lot of traffic is coming from ycombinator. So... (I mean, I don't have a point here but I find it pretty interesting. xD)

Duck Duck Go traffic is 10% of the Google traffic.

Is that typical for privacy-focused tech sites? I would have expected a lower percentage.

(DDG fan here, but people look at me funny when they see me using it...)

Re: How we built a GDPR-compliant website analytics platform without using cookies

#110

Post by hedora »

JackWritesCode wrote: post #105
tomp wrote: post #103

Maybe I'm missing something but (1) I don't think this is GDPR compliant, and (2) why so complicated? Regarding (1), > Brute forcing a 256 bit hash would cost 10^44 times the Gross World Product (GWP). [...] > We have rendered the data anonymous to the point where we could not identify a natural person from the hash. > It's possible that GDPR does not apply to Fathom since data is made completely anonymous. Even if G…

For GDPR compliance notes please see: https://usefathom.com/data/ > You don't need to "brute-force" the hash, you just need to find a user that matches your hash... which is 1 in 7 billion (or so), much more tractable. This is also the principle e.g. MD5 rainbow tables are based on... Not quite. We use a SHA256 hash as our salt, and that changes each day, so you'd need to brute force that. In terms of how many possib…

You can get an order of magnitude on hash collision resistance by rolling every two hours. Maintain “two” backend databases to gracefully track sessions between roll overs.

Also, for non GPDR IP blocks, maybe just store a per client salt in a cookie(!) and then xor it with the rotating server salt.

Post reply on HN