It should be released. Its the equivalent of stopping medical research because it could be used for biological weapons.
The people who need this for evil purposes will develop it whether it's released in open source or not.
“Should this even be released?” Deep learning tool that may be used for doxxing
31–40 of 64 posts
Re: “Should this even be released?” Deep learning tool that may be used for doxxing
#32Yes
Re: “Should this even be released?” Deep learning tool that may be used for doxxing
#33This should be released, if only so that we can learn how to evade it. Perhaps it could be broken by only writing anonymously in a different language from that of your public life, for example.
Re: “Should this even be released?” Deep learning tool that may be used for doxxing
#34Earlier quoted context omitted.
>"2. Being able to identify troublemakers in a community (such as a forum or a social networking site). I'm sure a lot of administrators would love to know if that suspicious looking new guy is the alias of a banned troll from a few weeks back (posting through a proxy server)." Orwellian.
So, what do you when a troll persistently and constantly attacks your community website? As in, flames everyone to a crisp, posts as much porn as possible, tries to incite a civil war between a few members that might not be on good terms with each other or the staff and registers hundreds of accounts, some of which stay semi dormant until they strike? Because that can happen very easily online, especially if you get…
In my day, we'd use moderators: put the forum/list in a mode where every new post goes through a human review first. After a while, things calm down, and you revert to the anything-goes default.
I don't know how well that would work for high-traffic forums of today, but it can scale pretty easily by employing many moderators.
Re: “Should this even be released?” Deep learning tool that may be used for doxxing
#35Yeah, it should be released. I mean sure, there are various 'immoral' uses for it (like doxxing), but there are also many good ones. Such as: 1. Working out who wrote a bunch of anonymous reviews on Amazon or other such sites, which could be used to stop fake reviews. You actually mention this usage in your article. 2. Being able to identify troublemakers in a community (such as a forum or a social networking site).…
Aren't these more or less the arguments used to justify anti-cryptography legislation? And when articles about banning encryption come up on here, people say pretty much the same thing: "Don't let the worries about its usage by 'bad' people overshadow the good". In fact I believe this tool is even more worrisome, because there are a very large number of non-tech savvy people who express their dissonant opinions simpl…
" Even though I believe the tool to be harmful, it's better to know that it exists, know its capabilities, and most importantly know how it works"
This is a really important point. Even if you don't like the possible uses of the tool, it's either "release it now and make it possible to defend against" or "don't release it, and hope the likes of the NSA don't develop their own version".
Re: “Should this even be released?” Deep learning tool that may be used for doxxing
#36This entire post and the debate surrounding it, is frankly stupid. What does 95% accuracy even means?? Consider face recognition, even when there is good gold standard for matching faces (human judgement, since human are good at recognizing faces), determining accuracy of Face recognition algorithms is still challenging (E.g. Megaface challenge). When it comes to a piece of text written by an author its even more dif…
> This entire post and the debate surrounding it, is frankly stupid. What does 95% accuracy even means??
Generally in supervised machine learning a claim of X% accuracy means that when tested on a large dataset for which the correct result is known and that was not part of the training dataset or validation dataset, it classified X% of that dataset correctly.
Typically you gather a big dataset of labeled data and then split it randomly into training, validation, and test sets. A 50/25/25 split is common. If the learning approach you are using does not need a validation set, then 70/30 training/test is common.
How reliable such an accuracy estimate is depends on how well your dataset matches the characteristics of the datasets people will be using your trained system on. His 95% accuracy report is probably reasonably reliable when his software is used on anonymous posts on the forums where he gathered his datasets. It would probably be less reliable looking at anonymous posts on, say, a bagpipe maker's forum.
Re: “Should this even be released?” Deep learning tool that may be used for doxxing
#37There already exists a counter tool to help against this kind of privacy invasion https://github.com/psal/anonymouth
I just tried to get anonymouth working, but unfortunately even after fixing the invalid code issues/errors preventing compilation, it crashes after you fill out about 6 screens worth questions about where various types of text content is located:
>>>>>>>>>>>>>>>>>>>>>>> LOGGING STACK TRACE Re: “Should this even be released?” Deep learning tool that may be used for doxxing
#38If there's a 5% false-positive rate, then if I give it an unidentified text and the posts from 1,000,000 identified Redittors, it's going to give me 50,000 possible authors? That doesn't seem either useful or troublesome...
That in and of itself would not be more than marginally useful or troublesome, but usually it won't be used by itself. Usually it will be combined with other evidence that you already have via Bayesian reasoning or intuition.
For example, suppose someone is revealing on Reddit details about some business dealing of yours that should have only been known by people who are under NDAs. If you intersect the set of 50000 Redditors returned by the deanonymizing tool with the set of people under your NDA, and that intersection is not empty, then the leaker is probably one of the ones in the intersection.
Re: “Should this even be released?” Deep learning tool that may be used for doxxing
#39I wonder if this could work when confronted with "hivemind" twitter accounts shared with lots of different people. I don't see how it could get enough signal from the noise to allow someone to unmask any identities.
Re: “Should this even be released?” Deep learning tool that may be used for doxxing
#40This entire post and the debate surrounding it, is frankly stupid. What does 95% accuracy even means?? Consider face recognition, even when there is good gold standard for matching faces (human judgement, since human are good at recognizing faces), determining accuracy of Face recognition algorithms is still challenging (E.g. Megaface challenge). When it comes to a piece of text written by an author its even more dif…
> This entire post and the debate surrounding it, is frankly stupid. What does 95% accuracy even means?? Generally in supervised machine learning a claim of X% accuracy means that when tested on a large dataset for which the correct result is known and that was not part of the training dataset or validation dataset, it classified X% of that dataset correctly. Typically you gather a big dataset of labeled data and the…
Huh... of course I know definition of accuracy, and how its calculated.
However even in supervised learning, accuracy is only used in very limited cases such as multi class classification. For a whole bunch of problems including the one being discussed its a poor and in some cases a biased metric. E.g. consider a heavily unbalanced problem 99% positive labels. By predicting all instances with majority label its possible to get 99% accuracy. There are several better metrics, False Accept rates, Precision Recall curves etc.
Without knowing how the dataset was collected, did the "username" leaked into the dataset, etc. its impossible to evaluate such outlandish claims.
The whole moral and ethical debate is non-sequitur, and harms legitimate deep learning research.