Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reproducing Hacker News writing style fingerprinting

antirez.com

Post reply on HN
Print view View original

61–70 of 168 posts

Re: Reproducing Hacker News writing style fingerprinting

#61

Post by ziddoap »

antirez wrote: post #60
ziddoap wrote: post #59

I noticed that in my top 20 similar users, the similarity rank/score/whatever are all >~0.83. However, randomly sampling from users in this thread, some top 20s are all Is there anything that can be inferred from that? Is my writing less unique, so ends up being more similar to more people? Also, someone like tptacek has a top 20 with matches all >0.87. Would this be a side-effect of his prolific posting, so matches…

It's not "less unique" as the structure of the sentence is what matters: the syntax. But you simply tend to use words with balanced frequency. It's not a bad thing.

Yeah, definitely not a bad thing. This just piqued my curiosity and is in a field I'm not super familiar with, so I'm just trying to wrap my head around it.

Thanks for the interesting tool!

Re: Reproducing Hacker News writing style fingerprinting

#62

Post by legohead »

Frieren wrote: post #33

It works for me. The accounts I used long time ago are there in high positions. I guess that my style is very distinctive. But I also have seen some accounts that seem to be from other non-native English speakers. They may even have a Latin language as their native one (I just read some of their comments, and, at minimum, some of them seem to also be from the EU). So, I guess, that it is also grouping people by their…

Didn't catch my original account when I tried it, not anywhere in top 100.

But, if I do the reverse (search using my original account), this one shows up as #2.

The main difference between the accounts is this one has a lot more posts, and my original account was actively posting ~11 years ago.

Re: Reproducing Hacker News writing style fingerprinting

#63

Post by shakna »

alganet wrote: post #46
shakna wrote: post #34

Earlier quoted context omitted.

I don't have other accounts, but still matched at 85+% accuracy for a half dozen accounts. Seems I don't have very original thoughts or writing style.

It's a fingerprinting tool, not a profiling tool. You can't draw such conclusions from it. What a profiler would do to identify someone, I imagine, requires much more. Like the ability to recognize someone's tendency of playing the victim to leverage social advantage in awkward situations.

85% is surprisingly high for fingerprinting, hence self-deprecation over insulting the author by poking at efficacy. I wouldn't have expected my Australian spelling, Oxford comma, or cadence to be anything close to the Californian Rust enthusiasts I apparently match against. Especially as there's no normalization happening - so even the Burrows-Delta method shouldn't match my use of "gaol" or "humour" that often.

But, limiting to the top couple hundred words, probably does limit me to sounding like a pretentious dickhole, as I often use "however", "but", and "isn't". Corrections are a little too frequent in my post history.

I'd expect things might be a tiny bit looser with precisions if something small like stop words were removed. Though, it'd be interesting to do the opposite. If you were only measuring stopwords, would that show a unique cadence?

Re: Reproducing Hacker News writing style fingerprinting

#64

Post by cratermoon »

antirez wrote: post #57
cratermoon wrote: post #36

Earlier quoted context omitted.

I noted the "analyze" feature didn't seem as useful as it could be because the majority of the words are common articles and conjunctions. I'd like to see a version of analyze that filters out at least the following stop words: a, an, and, are, as, at, be, but, by, for, if, in, into, is, it, no, not, of, on, or, such, that, the, their, then, there, these, they, this, to, was, will, with

The system uses on purpose those simple words, since they are "tellers" of the style of the user in a context independent way. Burrows papers explain this very well, but in general we want to capture low-level structure, more than topics and exact non obvious words used. I tested the system with 10k words and removing the most common words, and you get totally different results (still useful, but not style matching),…

>The system uses on purpose those simple words, since they are "tellers" of the style of the user in a context independent way.

Yes, that's good! I didn't state my interest clearly, though. I'd like to see the "analyze" result with the stop words excluded, not for the style comparison part, but for the reasons you state and others.

Re: Reproducing Hacker News writing style fingerprinting

#65

Post by wild_egg »

antirez wrote: post #55
wild_egg wrote: post #51

Very cool. Also a bit surprising — two of my matches are people I know IRL.

Are you all from the same town? Another user reported this finding.

We had all met in the same city but weren't originally from there or live there any longer.

Maybe some "like attracts like" phenomena

Re: Reproducing Hacker News writing style fingerprinting

#67

Post by yorwba »

antirez wrote: post #57
cratermoon wrote: post #36

Earlier quoted context omitted.

I noted the "analyze" feature didn't seem as useful as it could be because the majority of the words are common articles and conjunctions. I'd like to see a version of analyze that filters out at least the following stop words: a, an, and, are, as, at, be, but, by, for, if, in, into, is, it, no, not, of, on, or, such, that, the, their, then, there, these, they, this, to, was, will, with

The system uses on purpose those simple words, since they are "tellers" of the style of the user in a context independent way. Burrows papers explain this very well, but in general we want to capture low-level structure, more than topics and exact non obvious words used. I tested the system with 10k words and removing the most common words, and you get totally different results (still useful, but not style matching),…

I think grouping users by interests would be a more interesting application. Most users don't have multiple accounts, but everyone probably shares some interests with other users, whom they might enjoy discovering.

Re: Reproducing Hacker News writing style fingerprinting

#68

Post by alganet »

shakna wrote: post #63
alganet wrote: post #46

Earlier quoted context omitted.

It's a fingerprinting tool, not a profiling tool. You can't draw such conclusions from it. What a profiler would do to identify someone, I imagine, requires much more. Like the ability to recognize someone's tendency of playing the victim to leverage social advantage in awkward situations.

85% is surprisingly high for fingerprinting, hence self-deprecation over insulting the author by poking at efficacy. I wouldn't have expected my Australian spelling, Oxford comma, or cadence to be anything close to the Californian Rust enthusiasts I apparently match against. Especially as there's no normalization happening - so even the Burrows-Delta method shouldn't match my use of "gaol" or "humour" that often. But…

I don't know dude, don't take it personally.

Re: Reproducing Hacker News writing style fingerprinting

#69

Post by pc86 »

Frieren wrote: post #45
shakna wrote: post #34

Earlier quoted context omitted.

I don't have other accounts, but still matched at 85+% accuracy for a half dozen accounts. Seems I don't have very original thoughts or writing style.

My guess is that people from the same region and similar background will have more and closer "alters". So, if you are Californian-American then there is many people that will speak similar to you in HN. If you are a Satawalese speaker then you may be quite alone in your own group. (The Satawalese language has 460 speakers, most of who live in Satawal Island in the Federated States of Micronesia.)

You couldn't have just picked a European country, you had to flex on us with Satawalese? :)
Post reply on HN