Ask HN: Algorithms for text fingerprinting? • Hacker News Board

Re: Ask HN: Algorithms for text fingerprinting?

#21

Post by kngspook » Mon, Jun 15, 2015, 7:11 AM UTC

vixsomnis wrote: post #19

amazing_jose wrote: post #18

The results could be horrible, but can imagine a simple technique for hiding all those clues. Just send the text to google translate, translate it to an intermediate language and the back to the original one. I can warranty an excellent t rinse and clean. Change the intermediate language and you will change the features of the final text. Of course, you risk horrible semantic changes in the final text ;) UPDATE: fix…

Took only a minute to try: English -> Filipino (Tagalog) -> Chinese-simplified (Mandarin?) -> English I remember reading an article about a year ago (NSA) to identify the user, based on how they are written, vocabulary, spelling errors, grammar, language, and so on. It is interesting to me, because it is difficult to change the written and spoken word in use. It can be estimated that there are between two characters…

Google Translate almost holds it together...until the very end...

Original:

  In the beginning God created the heaven and the earth. And the earth was without form, and void; and darkness was upon the face of the deep. And the Spirit of God moved upon the face of the waters.

[a half dozen languages later...]

Result:

  God created heaven and earth in the beginning. And the earth was formless and empty, and darkness was on the face of the abyss, man. Answer the Spirit of God on the water surface.

Re: Ask HN: Algorithms for text fingerprinting?

#22

Post by ShaneWilton » Mon, Jun 15, 2015, 7:26 AM UTC

amazing_jose wrote: post #18

The results could be horrible, but can imagine a simple technique for hiding all those clues. Just send the text to google translate, translate it to an intermediate language and the back to the original one. I can warranty an excellent t rinse and clean. Change the intermediate language and you will change the features of the final text. Of course, you risk horrible semantic changes in the final text ;) UPDATE: fix…

There's a great paper that investigated this technique: https://www.eecs.berkeley.edu/~sa499/papers/adversarial_styl...

From the conclusions: "Translation with widely available machine translation services does not appear to be a viable mode of circumvention. Our evaluation did not demonstrate sufficient anonymization and the translated document has, at best, questionable grammar and quality."

Re: Ask HN: Algorithms for text fingerprinting?

#23

Post by espe » Mon, Jun 15, 2015, 7:28 AM UTC

afaik, stylo (https://sites.google.com/site/computationalstylistics/stylo) is the academic go-to solution. it is even sporting a gui

Re: Ask HN: Algorithms for text fingerprinting?

#24

Post by CSDude » Mon, Jun 15, 2015, 8:09 AM UTC

There is a public domain fingerprinting tool, that is used in MOSS (Measure Of Software Similarity). You can get the ideas from there.

http://theory.stanford.edu/~aiken/publications/papers/sigmod...

Re: Ask HN: Algorithms for text fingerprinting?

#25

Post by lucb1e » Mon, Jun 15, 2015, 8:24 AM UTC

nodelessness wrote: post #20

This one time JK Rowling was found out writing by a pseudonym[1] using this program: http://evllabs.com/jgaap/w/index.php/Main_Page [1] http://blogs.wsj.com/speakeasy/2013/07/16/the-science-that-u...

If I remember correctly, someone actually talked about it and they claimed using this program to cover for the person who leaked. But I may be wrong.

Edit: https://en.wikipedia.org/wiki/The_Cuckoo%27s_Calling#Authors...

> However, it was later reported that Rowling's authorship was leaked to a Times reporter via Twitter by the friend of the wife of a lawyer at Russells Solicitors, who had worked for Rowling. The firm has since apologised[29] and made a "substantial charitable donation" to the Soldiers' Charity as a result of legal action brought by Rowling.[30]

Re: Ask HN: Algorithms for text fingerprinting?

#26

Post by pdpd » Mon, Jun 15, 2015, 8:31 AM UTC

I wonder if this is how the authors of Truecrypt where identified. I remember reading something similar about this in regards to coding style.

I am sure the Truecrypt authors contributed to more than one project.

Re: Ask HN: Algorithms for text fingerprinting?

#27

Post by nl » Mon, Jun 15, 2015, 9:37 AM UTC

moyix wrote: post #9

The relevant search term is "stylometry". One particular paper I remember is from Dawn Song's group at Berkeley a couple years back: http://www.cs.berkeley.edu/~dawnsong/papers/2012%20On%20the%... There's a lot of public work on the topic, but it looks like right now the best place to look is still in academic papers (I don't know of any open source libraries, for example).

This is a good search to start from: https://scholar.google.com.au/scholar?as_ylo=2015&q=stylomet...

Re: Ask HN: Algorithms for text fingerprinting?

#28

Post by nl » Mon, Jun 15, 2015, 9:40 AM UTC

vixsomnis wrote: post #19

amazing_jose wrote: post #18

The results could be horrible, but can imagine a simple technique for hiding all those clues. Just send the text to google translate, translate it to an intermediate language and the back to the original one. I can warranty an excellent t rinse and clean. Change the intermediate language and you will change the features of the final text. Of course, you risk horrible semantic changes in the final text ;) UPDATE: fix…

Took only a minute to try: English -> Filipino (Tagalog) -> Chinese-simplified (Mandarin?) -> English I remember reading an article about a year ago (NSA) to identify the user, based on how they are written, vocabulary, spelling errors, grammar, language, and so on. It is interesting to me, because it is difficult to change the written and spoken word in use. It can be estimated that there are between two characters…

Google translate to Tagalog is generally not great.

If you chain through European languages you barely lose anything. Doing English->Dutch->German->English is a good set to use.

Re: Ask HN: Algorithms for text fingerprinting?

#29

Post by tokenizerrr » Mon, Jun 15, 2015, 10:21 AM UTC

vixsomnis wrote: post #19

amazing_jose wrote: post #18

The results could be horrible, but can imagine a simple technique for hiding all those clues. Just send the text to google translate, translate it to an intermediate language and the back to the original one. I can warranty an excellent t rinse and clean. Change the intermediate language and you will change the features of the final text. Of course, you risk horrible semantic changes in the final text ;) UPDATE: fix…

Took only a minute to try: English -> Filipino (Tagalog) -> Chinese-simplified (Mandarin?) -> English I remember reading an article about a year ago (NSA) to identify the user, based on how they are written, vocabulary, spelling errors, grammar, language, and so on. It is interesting to me, because it is difficult to change the written and spoken word in use. It can be estimated that there are between two characters…

Of course you're sending all this information to Google now. Are there any offline translators that are advanced enough to be used for something like this? I imagine most just naively map words 1:1 which wouldn't do much good here.

Re: Ask HN: Algorithms for text fingerprinting?

#30

Post by gnur » Mon, Jun 15, 2015, 10:36 AM UTC

I've started a simular project myself recently, I check on various parameters (reading level score, words per sentence, syllables per word, sentences per paragraph, average word length, average syllable count) and calculate the distance between 2 texts / authors using a simple euclidean distance.

I started out with the code provided on https://github.com/mac389/ToxTweet/blob/master/textanalyzer.... I use it in a private project, but the results are promising!