“Should this even be released?” Deep learning tool that may be used for doxxing • Hacker News Board

Re: “Should this even be released?” Deep learning tool that may be used for doxxing

#21

Post by DeadHitchhiker » Sat, Jul 30, 2016, 7:57 PM UTC

CM30 wrote: post #14

a_small_island wrote: post #8

Earlier quoted context omitted.

>"2. Being able to identify troublemakers in a community (such as a forum or a social networking site). I'm sure a lot of administrators would love to know if that suspicious looking new guy is the alias of a banned troll from a few weeks back (posting through a proxy server)." Orwellian.

So, what do you when a troll persistently and constantly attacks your community website? As in, flames everyone to a crisp, posts as much porn as possible, tries to incite a civil war between a few members that might not be on good terms with each other or the staff and registers hundreds of accounts, some of which stay semi dormant until they strike? Because that can happen very easily online, especially if you get…

Ban behavior, not people. You can convert non-productive people to productive people, most of them only want to be noticed or accepted and past a certain point there is only so much you can do to block anyone anyway. Stylometric analysis would just be another simple hurdle to cross for a persistent person.

The idea that this tool would be useful for community management is terrible.

Re: “Should this even be released?” Deep learning tool that may be used for doxxing

#22

Post by wrs » Sat, Jul 30, 2016, 7:59 PM UTC

If there's a 5% false-positive rate, then if I give it an unidentified text and the posts from 1,000,000 identified Redittors, it's going to give me 50,000 possible authors? That doesn't seem either useful or troublesome...

Re: “Should this even be released?” Deep learning tool that may be used for doxxing

#23

Post by mikeleeorg » Sat, Jul 30, 2016, 7:59 PM UTC

sparkie wrote: post #18

There already exists a counter tool to help against this kind of privacy invasion https://github.com/psal/anonymouth

I was just going to ask if such a counter tool existed already. Wouldn't be too hard to put a web UI to this and make it easily accessible. Thanks for sharing.

Re: “Should this even be released?” Deep learning tool that may be used for doxxing

#24

Post by diyseguy » Sat, Jul 30, 2016, 8:00 PM UTC

People tend to think and write in the same familiar memes and patterns - I doubt very much it could discern individuals based on text samples.

Re: “Should this even be released?” Deep learning tool that may be used for doxxing

#25

Post by FLUX-YOU » Sat, Jul 30, 2016, 8:07 PM UTC

CM30 wrote: post #5

Yeah, it should be released. I mean sure, there are various 'immoral' uses for it (like doxxing), but there are also many good ones. Such as: 1. Working out who wrote a bunch of anonymous reviews on Amazon or other such sites, which could be used to stop fake reviews. You actually mention this usage in your article. 2. Being able to identify troublemakers in a community (such as a forum or a social networking site).…

I'd guess if it can be used to identify you, it can be used to adjust your new posts until they don't appear to be written by you: for a practical example, a Chrome addon that detects when you are making social media posts and automatically flags identifying information like a spellcheck does. So the thing can be used to counter itself.

Re: “Should this even be released?” Deep learning tool that may be used for doxxing

#26

Post by noonespecial » Sat, Jul 30, 2016, 8:08 PM UTC

a_small_island wrote: post #8

CM30 wrote: post #5

Yeah, it should be released. I mean sure, there are various 'immoral' uses for it (like doxxing), but there are also many good ones. Such as: 1. Working out who wrote a bunch of anonymous reviews on Amazon or other such sites, which could be used to stop fake reviews. You actually mention this usage in your article. 2. Being able to identify troublemakers in a community (such as a forum or a social networking site).…

>"2. Being able to identify troublemakers in a community (such as a forum or a social networking site). I'm sure a lot of administrators would love to know if that suspicious looking new guy is the alias of a banned troll from a few weeks back (posting through a proxy server)." Orwellian.

That's especially troubling because the easiest and cheapest thing for any commercial service to do is to ban forever and without appeal any user that presents a hint of trouble to the algorithm. If these services have near monopolies, or share data with all of the other services like CLUE does for insurance... Its like a digital death sentence.

Ask anyone who's ever lost an ebay or a google account to an algorithmic burp and was essentially banned for life without appeal or even human oversight.

Re: “Should this even be released?” Deep learning tool that may be used for doxxing

#27

Post by fpgaminer » Sat, Jul 30, 2016, 8:11 PM UTC

CM30 wrote: post #5

Yeah, it should be released. I mean sure, there are various 'immoral' uses for it (like doxxing), but there are also many good ones. Such as: 1. Working out who wrote a bunch of anonymous reviews on Amazon or other such sites, which could be used to stop fake reviews. You actually mention this usage in your article. 2. Being able to identify troublemakers in a community (such as a forum or a social networking site).…

Aren't these more or less the arguments used to justify anti-cryptography legislation? And when articles about banning encryption come up on here, people say pretty much the same thing: "Don't let the worries about its usage by 'bad' people overshadow the good".

In fact I believe this tool is even more worrisome, because there are a very large number of non-tech savvy people who express their dissonant opinions simply under the mask imparted by internet anonymity. I imagine most everyone here on HN have at least once made an anonymous account to post a comment somewhere that they would rather not have tied to their identity. This is an avenue that is necessary for the preservation of free speech. Remember that things like treating black people as equals, giving women the right to vote, gay rights, etc were and in some ways still are taboo subjects that bring the wrath and ire of the power du jure.

All that said, I still don't see any reason why such a tool shouldn't be released. Why? Even though I believe the tool to be harmful, it's better to know that it exists, know its capabilities, and most importantly know how it works. It'll end up in the hands of the wrong people anyway, so it's better to at least get it into the hands of the right people who can possible do something to combat it.

Re: “Should this even be released?” Deep learning tool that may be used for doxxing

#28

Post by Animats » Sat, Jul 30, 2016, 8:11 PM UTC

There are papers on how to do this, and many approaches work. Traditional statistical methods[1], support vector machines,[2][3] (software available at [4]) and a random forest algorithm [5] have been shown to work, more or less. This isn't a new idea. All this new code does is let us compare how deep learning does on the problem.

[1] https://www.aclweb.org/anthology/E/E99/E99-1021.pdf [2] http://ceur-ws.org/Vol-1391/126-CR.pdf [3] https://www.aaai.org/ocs/index.php/FLAIRS/FLAIRS13/paper/vie... [4] http://www.cs.waikato.ac.nz/ml/weka/ [5] http://ntv.ifmo.ru/en/article/15185/kompyuternaya_kriminalis...

Re: “Should this even be released?” Deep learning tool that may be used for doxxing

#29

Post by firebones » Sat, Jul 30, 2016, 8:13 PM UTC

tiredofnoobs wrote: post #11

This entire post and the debate surrounding it, is frankly stupid. What does 95% accuracy even means?? Consider face recognition, even when there is good gold standard for matching faces (human judgement, since human are good at recognizing faces), determining accuracy of Face recognition algorithms is still challenging (E.g. Megaface challenge). When it comes to a piece of text written by an author its even more dif…

Agree. Rather than release the code, release the testing data set and/or test methodology used to make a claim of 95% accuracy; no one is harmed and it should give a good sense of the claim. I wouldn't be shocked to find that the score was achieved by overtraining on a very small and narrow data set (small number of identities), and that the model isn't generalizable.

That said, identification may be a more tractable problem if you have a limited population, additional metadata for features, and normalized writing samples (comparing anonymous reviews to identified reviews, or within community posts, as opposed to trying to compare a set of anonymous tweets to an identifiable dissertation).

Re: “Should this even be released?” Deep learning tool that may be used for doxxing

#30

Post by abstractbeliefs » Sat, Jul 30, 2016, 8:13 PM UTC

CM30 wrote: post #14

a_small_island wrote: post #8

Earlier quoted context omitted.

>"2. Being able to identify troublemakers in a community (such as a forum or a social networking site). I'm sure a lot of administrators would love to know if that suspicious looking new guy is the alias of a banned troll from a few weeks back (posting through a proxy server)." Orwellian.

So, what do you when a troll persistently and constantly attacks your community website? As in, flames everyone to a crisp, posts as much porn as possible, tries to incite a civil war between a few members that might not be on good terms with each other or the staff and registers hundreds of accounts, some of which stay semi dormant until they strike? Because that can happen very easily online, especially if you get…

Do we really need machine learning deanonymisation tools to identify trolls so unsubtle and obvious?

I voted to release the tool, but you've set up this false scenario with the intention of knocking it down easily and discrediting the opposing argument.