Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we built a GDPR-compliant website analytics platform without using cookies

usefathom.com

Post reply on HN
Print view View original

111–120 of 129 posts

Re: How we built a GDPR-compliant website analytics platform without using cookies

#111

Post by JackWritesCode »

hedora wrote: post #110
JackWritesCode wrote: post #105

Earlier quoted context omitted.

For GDPR compliance notes please see: https://usefathom.com/data/ > You don't need to "brute-force" the hash, you just need to find a user that matches your hash... which is 1 in 7 billion (or so), much more tractable. This is also the principle e.g. MD5 rainbow tables are based on... Not quite. We use a SHA256 hash as our salt, and that changes each day, so you'd need to brute force that. In terms of how many possib…

You can get an order of magnitude on hash collision resistance by rolling every two hours. Maintain “two” backend databases to gracefully track sessions between roll overs. Also, for non GPDR IP blocks, maybe just store a per client salt in a cookie(!) and then xor it with the rotating server salt.

I’m not worried about hash collision with sha256. Any reason why I should be? :)

And we can’t use cookies because of PECR!

Re: How we built a GDPR-compliant website analytics platform without using cookies

#112

Post by number6 »

JackWritesCode wrote: post #86
number6 wrote: post #81

Earlier quoted context omitted.

Article 4 (1) states: ‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, me…

> Online identifiers for profiling and identification > Natural persons may be associated with online identifiers provided by their devices, applications, tools and protocols, such as internet protocol addresses, cookie identifiers or other identifiers such as radio frequency identification tags. > This may leave traces which, in particular when combined with unique identifiers and other information received by the s…

If you dont need the identifier, why don't leave it out at all (Art 5c)? Or is just in it this case unique?

Re: How we built a GDPR-compliant website analytics platform without using cookies

#113

Post by krageon »

orra wrote: post #75
lmkg wrote: post #9

Earlier quoted context omitted.

Hi Paul, thanks for being open about this. I have a big, important question. ICO, the agency in charge of enforcing GDPR and related legislation in England, released guidance earlier this month on the topics of cookies. One of the most notable parts of this guidance is that "device fingerprinting" is treated the same as a cookie[1]. And also that website analytics requires consent to use cookies or similar technologi…

I think that's a fair question. AFAICT, v1 of PECR awkwardly applies whenever the cookie is not functionally directly necessary for the service that the user is using. PECR applies even if, like here, the cookie is just for counting unique numbers of visitors, and is not used for fingerprinting individuals. The draft v2 of PECR contains an exemption for first party analytics. I think this maybe strikes a nice balance…

What makes analytics first-party? When the first party serves them, or only if the data never leaves machines under the direct and exclusive control of the first party?

Re: How we built a GDPR-compliant website analytics platform without using cookies

#114

Post by cameronbrown »

JackWritesCode wrote: post #15
abtestdev wrote: post #8

Earlier quoted context omitted.

Based on the blog - anyone who shares a IP address (such as inside a company network) would effectively look the same.

Indeed. This is one of the limitations to our approach. We're thinking of ways to workaround it. Please shout if you have any ideas ;)

If/When IPv6 is widely used the problem should fade away.

Re: How we built a GDPR-compliant website analytics platform without using cookies

#115

Post by i_anon »

Hi Jack and Paul - love what you're doing! This solution is so needed.

I wondered whether you could explain what makes your hashing different from the hashing used by Facebook for their custom audiences tool which was deemed unsuitable for anonymisation as per https://www.spiritlegal.com/en/news/details/e-commerce-retai...

Re: How we built a GDPR-compliant website analytics platform without using cookies

#116

Post by i_anon »

jfk13 wrote: post #27
JackWritesCode wrote: post #12

Earlier quoted context omitted.

Thanks for the concern here. We are GDPR compliant (and may be exempt from it). See here: https://usefathom.com/data/

You might like to edit the line on that policy page that refers to "the most privacy-focused manor"... while a privacy-focused manor is an interesting idea, I suspect you meant "manner". :)

Equally, I'm not sure what it means to be GDPR "complaint" but I'm thinking it's probably supposed to be "compliant" ;)

Re: How we built a GDPR-compliant website analytics platform without using cookies

#118

Post by JackWritesCode »

number6 wrote: post #112
JackWritesCode wrote: post #86

Earlier quoted context omitted.

> Online identifiers for profiling and identification > Natural persons may be associated with online identifiers provided by their devices, applications, tools and protocols, such as internet protocol addresses, cookie identifiers or other identifiers such as radio frequency identification tags. > This may leave traces which, in particular when combined with unique identifiers and other information received by the s…

If you dont need the identifier, why don't leave it out at all (Art 5c)? Or is just in it this case unique?

Identifier is used for previous view only. Previous view is updated when a new view is inserted into the temp table & previous view's user identifier is wiped.

Re: How we built a GDPR-compliant website analytics platform without using cookies

#119

Post by nybble41 »

JackWritesCode wrote: post #102
nybble41 wrote: post #100

Earlier quoted context omitted.

It's true that you'd need to know the salt first, but the server does know the salt, and if you obtained the hash through a data breach then you probably obtained the current day's salt as well.

So if someone had unlimited access to our servers, that would be a problem. One piece to note is that page views get deleted after around 30 minutes. So one of the reasons we posted to HN was for conversations like this. Reading what you put makes me think we need to do more when creating the PageRequestSignature and the SiteRequestSignature, because if someone had access to our server, got the hash, stopped all cron…

The case I would be (slightly) concerned about would be where an attacker has obtained limited, read-only access to your database, so they have the current salt and the hashes related to the last 30 minutes of activity but not the ability to simply log whatever data they want. At that point brute-forcing the 32-bit address space and a small set of common user agent strings would allow them to determine the IP addresses for specific page views.

Of course, if an attacker has access to the salt and is interested in the pages viewed by a specific known IP address then this all becomes much simpler. Then the only unknown is the user agent, which is relatively low-entropy.

> So we'd not recycle a single salt, we'd recycle multiple salts that are based on, perhaps, the first 2-3 digits of a users IP address combined with the last 2-3 digits....

If an attacker is already brute forcing the IP address and has the ability to obtain the salt then they would just use the correct salt for each IP address. Making the salt depend on other data already included in the hash doesn't change the size of the search space.

Re: How we built a GDPR-compliant website analytics platform without using cookies

#120

Post by gcbw2 »

JackWritesCode wrote: post #101
gcbw2 wrote: post #99

This is still logging everything the GDPR says you can't without asking for consent, but you made your search convoluted (but not less efficient if you have all the pieces) to (suggest|lie?) that you need to break the hash and that's why you don't need consent. None of the information you are using on the hash wouldn't be in the search query itself! ip, user agent, path, date, etc. So there is no way to reverse the h…

You're focused mostly on Recital 26, which was only a theory of mine, outside of that we are GDPR compliant anyway. I likely shouldn't have included it since that isn't our primary ground for processing. Please see: https://usefathom.com/data/ And yes the daily hash gets stored until midnight. But what are you talking about with 'search query' containing IP, user agent etc.?

If a search query on your data would contain all the components of the original hash, i don't have to walk backwards and break the hash. i just have to hash my query terms in the same way.

Also I suggested you store the daily hash forever. But even if you really erase it every day, as you say, If you or an attacker makes the same request every day at a predetermined time, when you/they get your logs, you/they can use that predictable request to get the daily secret too.

I consider the information to be stored in plain text, and that you would have to have requested permission just the same. You pretty much have an identifiable user (via IP/UA/access time) stored in your logs.

Anonymization is removal of information, not encoding it in a convoluted hash.

Post reply on HN