Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large-Scale Online Deanonymization with LLMs

simonlermen.substack.com

Post reply on HN
Print view View original

41–50 of 258 posts

Re: Large-Scale Online Deanonymization with LLMs

#41

Post by joe_mamba »

yomismoaqui wrote: post #38

I did something like this passing some of my comments here and then prompted Gemini to identify my native language by reading my not-so-good english. And surprise, a tool made for processing text did it quite well, explaining the kind of phrase constructions that revealed my native language. So maybe this is a plus for passing any text published on the internet through a slopifier for anonymization? EDIT: deanonymiza…

>So maybe this is a plus for passing any text published on the internet through a slopifier for deanonymization?

Or vice versa, Indian scammers online can now run their traditional Victorian English phrasing through an AI to sound more authentically American.

Interviewers now have to deal with remote North Korean deepfaked candidates pretending to be Americans.

Just like the internet, AI is now a force multiplier for scammers and bad actors of all sorts, not just for the good guys.

Re: Large-Scale Online Deanonymization with LLMs

#45

Post by palmotea »

mhitza wrote: post #3

i haven't read the full study, but its been on my mind for a while. https://en.wikipedia.org/wiki/Stylometry The best course of action to combat this correlation/profiling, seems to be usage of a local llm that rewrites the text while keeping meaning untouched. Ideally built into a browser like Firefox/Brave.

> The best course of action to combat this correlation/profiling, seems to be usage of a local llm that rewrites the text while keeping meaning untouched.

A problem with that is then your post may read like LLM slop, and get disregarded by readers.

Another reason why LLMs are destruction machines.

Re: Large-Scale Online Deanonymization with LLMs

#46

Post by DalasNoin »

ranger_danger wrote: post #23
DalasNoin wrote: post #4

Earlier quoted context omitted.

To be clear, we are making a clear concession here that the people weren't truly anonymous. But we did use an LLM to remove any identifying information from HN making them quasi-anonymous, this is more described in the appendix Table 2. We do also make a more real world like test in section 2. There we use the anthropic interviewer dataset which Anthropic redacted, from the redacted interviews our agent identified 9/…

But you also relied on people giving away too much personal information about themselves... which won't always be the case.

I agree that these accounts probably on average still contain more information than the average pseudonymous account. I think we could try to use the LLM to increasingly ablate more information and see how it performance decays – to be clear we already heavily remove such information, see Table 2 appendix. But I don't expect that to change the basic conclusions.

Re: Large-Scale Online Deanonymization with LLMs

#47

Post by intended »

kseniamorph wrote: post #30

I'm not sure the practical implications are as dramatic as the paper suggests. Most adversaries who would want to deanonymize people at scale (governments, corporations) already have access to far more direct methods. The people most at risk from this are probably activists and whistleblowers in jurisdictions where those direct methods aren't available, not average users.

People who comment about their boss and workplaces?

People on HN who talk about their work but want to remain anonymous? People who don’t want to be spammed if they comment in a community? Or harassed if they comment in a community? Maybe someone doesn’t want others to find out they are posting in r/depression. (Or r/warhammer.)

Anonymity is a substantial aspect of the current internet. It’s the practical reason you can have a stance against age verification.

On the other hand, if anonymity can be pierced with relative ease, then arguments for privacy are non sequiturs.

Re: Large-Scale Online Deanonymization with LLMs

#48

Post by john_strinlai »

many people tend to overlook how little information is needed for successful de-anonymization.

i like to introduce students to de-anonymization with an old paper "Robust De-anonymization of Large Sparse Datasets" published in the ancient history of 2008 (https://www.cs.cornell.edu/~shmat/shmat_oak08netflix.pdf):

"We apply our de-anonymization methodology to the Netflix Prize dataset, which contains anonymous movie ratings of 500,000 subscribers of Netflix [...]. We demonstrate that an adversary who knows only a little bit about an individual subscriber can easily identify this subscriber’s record in the dataset."

and that was 20 years ago! de-anonymization techniques have improved by leaps and bounds since then, alongside the massive growth in various technology that enhances/enables various techniques.

i think the age of (pseduo-)anonymous internet browsing will be over soon. certainly within my lifetime (and im not that young!). it might be by regulation, it might be by nature of dragnet surveillance + de-anonymization, or a combination of both. but i think it will be a chilling time.

Re: Large-Scale Online Deanonymization with LLMs

#49

Post by graemep »

kseniamorph wrote: post #30

I'm not sure the practical implications are as dramatic as the paper suggests. Most adversaries who would want to deanonymize people at scale (governments, corporations) already have access to far more direct methods. The people most at risk from this are probably activists and whistleblowers in jurisdictions where those direct methods aren't available, not average users.

I can imagine a lot of countries who want to control what their citizens say abroad. I know Iraq in Saddam Hussein's time did it in the UK, China does it now.

Re: Large-Scale Online Deanonymization with LLMs

#50

Post by john_strinlai »

intended wrote: post #47
kseniamorph wrote: post #30

I'm not sure the practical implications are as dramatic as the paper suggests. Most adversaries who would want to deanonymize people at scale (governments, corporations) already have access to far more direct methods. The people most at risk from this are probably activists and whistleblowers in jurisdictions where those direct methods aren't available, not average users.

People who comment about their boss and workplaces? People on HN who talk about their work but want to remain anonymous? People who don’t want to be spammed if they comment in a community? Or harassed if they comment in a community? Maybe someone doesn’t want others to find out they are posting in r/depression. (Or r/warhammer.) Anonymity is a substantial aspect of the current internet. It’s the practical reason you…

another big one: people looking for insurance, or looking to claim insurance
Post reply on HN