Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

US Government plans to develop AI that can unmask anonymous writers

reclaimthenet.org

Post reply on HN
Print view View original

181–190 of 219 posts

Re: US Government plans to develop AI that can unmask anonymous writers

#181

Post by CTDOCodebases »

boarnoah wrote: post #25
lwneal wrote: post #17

The best protection against this type of de-anonymization is to take measures now, while you still have time, to prevent it. It is possible to change the style of one's writing by using a language model which alters the original text in order to create a new piece with a different style. For example, to translate your text into the grandiose and flowing diction of a bygone era, you might consider the project below. […

There is a case to be made, not just for natural language but code. AFAIK there is quite a bit of examples from security labs where malware authors aren't necessarily identified but at least fingerprinted based on naming conventions, patterns they use across multiple projects etc... That sort of fingerprinting could expand to correlating someone's anonymous software projects to other examples of code elsewhere (ex: i…

I suspect this is why TrueCrypt shut down.

A paper got released in 2015 that claimed 94% accuracy in identifying authors.

I’m sure it would have been quite easy for the NSA to figure out who Satoshi Nakamoto was too considering PRISM was also scooping up everyone’s email.

Re: US Government plans to develop AI that can unmask anonymous writers

#182

Post by Al-Khwarizmi »

Shorel wrote: post #156
noir_lord wrote: post #108

Earlier quoted context omitted.

People reading what I write are frequently users of English as a second language. Expecting most of the English people to know obviate is pushing it, for someone using English as a second language it’s just adding complexity. It’s about aiming where you expect the average reader to be. Because you know the word don’t fall into the trap of assuming everyone else does.

Many times it's the other way around. Obviate has several cognates in romance languages, so it is straightforward. Now, the phrasal verbs that are so obvious to a native speaker, these are the real head-scratchers. Or street slang. It may sound like dumbing it down to you, but it actually adds complexity for a non-native.

Indeed.

"Register": I immediately know what it means and it takes tenths of a second for me to make the decision to click (or not to).

"Sign up": despite having a CEFR C2 level of English and using it at work every day, I still need to think for a few seconds if this is a registration or a login (cf. "sign in").

For a native speaker I suppose the latter is (marginally?) easier, but for a non-native, it's much harder, not even close.

Re: US Government plans to develop AI that can unmask anonymous writers

#183

Post by vixen99 »

noir_lord wrote: post #96
dehrmann wrote: post #8

Plans? I assumed lots of people were already working on this. There's already a lot of training data out there, and I suspect most users can be identified by use of a handful of uncommon trigrams and sentence stats. I know you can recognize things I've written at work because they have real em dashes—people rarely type with those.

You can probably spot me at work because I use really simple words. I write obviate then I re-read and replace it with “removes the need to do $foo. Some writers at work forget that the goal of the written word is to communicate to the reader not impress them with word salad. I regard it the same as clever code (note: not the same as complex code) - written for the author not the reader.

Nit pick. Fair point in some ways but it depends on your audience. If the latter consists primarily of folk with an extensive vocabulary that enables them to detect the nuance that might be intended by the use of a supposedly unusual word (like obviate) then they won't know what you mean by 'word salad' unless you're using some word inappropriately. Seven letters versus twenty three characters. No competition. If obviate is what you mean then use it. For a general audience however, you're absolutely right.

Occurs to me there's another point in your favor as expressed in Orwell's version of a well-known passage from the Old Testament 'Ecclesiastes' into modern English. He makes the point brilliantly. Aesthetics does come into it!

https://www.orwellfoundation.com/the-orwell-foundation/orwel...

Re: US Government plans to develop AI that can unmask anonymous writers

#184

Post by kleene_op »

ergodic1 wrote: post #165
black_puppydog wrote: post #157

As someone who expects this to have happened years ago (maybe not under the moniker of AI but who cares) I'm more shocked by the fact that they'd publically announce this. The chilling effects of this will be all too real. If this works it's pretty much the equivalent of a mandatory state ID on every online interaction. If it doesn't work very well, then it's going to be that, plus the risk of randomly being flagged.…

Won't there be a countermeasure? A grammarly for privacy that takes your intent and outputs an un-anonymizable text?

I can think of a few.

An offensive one would be to post fake offers to trade government/military intelligence on the dark web writen in the style of the politicians backing up this measure, so that they are put on the feds list and thoroughly investigated.

Re: US Government plans to develop AI that can unmask anonymous writers

#185

Post by geysersam »

black_puppydog wrote: post #157

As someone who expects this to have happened years ago (maybe not under the moniker of AI but who cares) I'm more shocked by the fact that they'd publically announce this. The chilling effects of this will be all too real. If this works it's pretty much the equivalent of a mandatory state ID on every online interaction. If it doesn't work very well, then it's going to be that, plus the risk of randomly being flagged.…

> If this works

Seems like an impossible task. There's just not enough signal to noise ratio to make that distinction. Not if the goal is to distinguish among thousands of people.

Among tens or hundreds, might work if they don't take any countermeasures.

Re: US Government plans to develop AI that can unmask anonymous writers

#186

Post by geysersam »

dehrmann wrote: post #8

Plans? I assumed lots of people were already working on this. There's already a lot of training data out there, and I suspect most users can be identified by use of a handful of uncommon trigrams and sentence stats. I know you can recognize things I've written at work because they have real em dashes—people rarely type with those.

How robust is it though? If you scale to thousands or tens of thousands of people. If it depends on trigrams and uncommon characters it can be countered by incredibly simple measures.

Re: US Government plans to develop AI that can unmask anonymous writers

#187

Post by geysersam »

canjobear wrote: post #127
_8j50 wrote: post #10

Earlier quoted context omitted.

NSA already does this. "Stylometry" I believe is the term. Perhaps they used heuristics and algorithms so far? I would NLP was good enough for this years ago.

I taught this as a routine class project in 2018.

What training data did you use in that project?

Re: US Government plans to develop AI that can unmask anonymous writers

#188

Post by gonzo41 »

Ok, so would the simplest way to solve this be an open source AI that can normalize specific text / writing to force it to be too similar to all other text so it's less identifiable.

Such as, you write a comment or an essay, feed it in and it just dumps all your styles, idioms etc and makes all your stuff sound bland and normal.

Seems like it would only work if you had a targeted population and a writing / style sample ffor a lot of people.

Re: US Government plans to develop AI that can unmask anonymous writers

#189

Post by classified »

geysersam wrote: post #185
black_puppydog wrote: post #157

As someone who expects this to have happened years ago (maybe not under the moniker of AI but who cares) I'm more shocked by the fact that they'd publically announce this. The chilling effects of this will be all too real. If this works it's pretty much the equivalent of a mandatory state ID on every online interaction. If it doesn't work very well, then it's going to be that, plus the risk of randomly being flagged.…

> If this works Seems like an impossible task. There's just not enough signal to noise ratio to make that distinction. Not if the goal is to distinguish among thousands of people. Among tens or hundreds, might work if they don't take any countermeasures.

Even if it doesn't exactly work, like with facial recognition fucking up for non-white skin colors, they will use it anyway and real people will be saddled with real adverse consequences.

Re: US Government plans to develop AI that can unmask anonymous writers

#190

Post by christophilus »

black_puppydog wrote: post #157

As someone who expects this to have happened years ago (maybe not under the moniker of AI but who cares) I'm more shocked by the fact that they'd publically announce this. The chilling effects of this will be all too real. If this works it's pretty much the equivalent of a mandatory state ID on every online interaction. If it doesn't work very well, then it's going to be that, plus the risk of randomly being flagged.…

> plus the risk of randomly being flagged

This is 100% going to happen, so my guess is it can’t be authoritative. It’s likely to be used to whittle down a list so that humans can review the results.

Feels like the beginning of a hybrid Minority Report + Enemy of the State movie. I’d watch that. I don’t want to live in that world, though.

Post reply on HN