Live data from Hacker News

Finding Sociopaths on Facebook

schneier.com

11–20 of 73 posts

Re: Finding Sociopaths on Facebook

#12
post #2

An excellent post by Schneier. > The problem isn't just that such a system is wrong, it's that the mathematics of testing makes this sort of thing pretty ineffective in practice. It's called the "base rate fallacy." Suppose you have a test that's 90% accurate in identifying both sociopaths and non-sociopaths. If you assume that 4% of people are sociopaths, then the chance of someone who tests positive actually being…

I found his version clearer because "%" distinguishes proportions from quantities.

Re: Finding Sociopaths on Facebook

#13
The "fallacy" described here is a non-issue. Medical tests have the same characteristics but are still incredibly useful. HIV testing is roughly 99% accurate, but given the large number of tests performed, a very large percentage of people will end up with the wrong result. And yet this test is vital.

Even wildly inaccurate tests can be useful. Imagine that driving drunk will result in a accident 10% of the time. 90% of the time, however, a driver will make it home safely. This is a wildly inaccurate predictor, but it still critical to know someone's blood alcohol level before giving them their keys.

One must simply be aware of the uncertainty involved in any test, and treat test results as probabilistic signal, not as proof.

Re: Finding Sociopaths on Facebook

#14
post #6

Earlier quoted context omitted.

This is a pretty bad example to illustrate the fallacy because a 25% confidence is actually extremely good. I don't know what good a test for sociopathy is, but if we had a test this good at identifying terrorists, it would be incredibly useful. If signals intelligence could produce a list of people and guarantee that a quarter of the people on that list are terrorists, it would absolutely revolutionize law enforceme…

> and guarantee that a quarter of the people on that list are terrorists, Honest question, what would happen to the other three quarters?

They'd be subject to background checks and surveillance that they ideally would not even notice - yes, I'm aware that we don't live in an ideal world, but the point is that traditional "leg work" policing is pretty good at determining whether a suspect is actually engaged in nefarious activities - but it's expensive and requires a reasonably narrow list of suspects to begin with.

Re: Finding Sociopaths on Facebook

#15
post #13

The "fallacy" described here is a non-issue. Medical tests have the same characteristics but are still incredibly useful. HIV testing is roughly 99% accurate, but given the large number of tests performed, a very large percentage of people will end up with the wrong result. And yet this test is vital. Even wildly inaccurate tests can be useful. Imagine that driving drunk will result in a accident 10% of the time. 90%…

The issue is that civil rights are supposed to be immune to probabilistic signal. In practice, it won't turn out that way. Look at racial profiling.

Re: Finding Sociopaths on Facebook

#16
post #13

The "fallacy" described here is a non-issue. Medical tests have the same characteristics but are still incredibly useful. HIV testing is roughly 99% accurate, but given the large number of tests performed, a very large percentage of people will end up with the wrong result. And yet this test is vital. Even wildly inaccurate tests can be useful. Imagine that driving drunk will result in a accident 10% of the time. 90%…

I tend to agree with you, except it is not a non issue.

Look at the numbers of people getting very serious medical treatment because they, and their clinicians, have not understood the numbers.

If doctor (well educated intelligent person) cannot get this right I'm scared that labelling someone as "POTENTIAL TERRORIST" on the basis of a 1 in X possibility is going to have disastrous consequences.

Re: Finding Sociopaths on Facebook

#18
post #6

Earlier quoted context omitted.

This is a pretty bad example to illustrate the fallacy because a 25% confidence is actually extremely good. I don't know what good a test for sociopathy is, but if we had a test this good at identifying terrorists, it would be incredibly useful. If signals intelligence could produce a list of people and guarantee that a quarter of the people on that list are terrorists, it would absolutely revolutionize law enforceme…

> and guarantee that a quarter of the people on that list are terrorists, Honest question, what would happen to the other three quarters?

Most likely nothing. It depends on what sort of test it is; if it's something non-intuitive like (say) a habit of writing sentences that always have a prime number of words in them, you'll get your false positives but most those people won't pass any other tests, whereas the actual terrorists will.

What Schneier is missing is that while you can't ID people that well from a single test, you can apply a bunch of them. In his example, one test improves the probability of correctly ID a sociopath from 4% to 24%. Apply another, different test of similar efficacy to that result set and you'll have a population of 21 true positives, and 8 or 9 false positives, increasing the probabiliy of a successful ID from 25% to ~70%. Sure, there's no single test that will give you reliable answers, but so what? It's OK to use a multi-pronged solution.

Re: Finding Sociopaths on Facebook

#19
post #9

I really doubt sociopaths would use facebook at all anyway

Perhaps you’re thinking of people who are antisocial? I wouldn’t be surprised if some of the Facebook members who are most active and have the most ‘friends’ are sociopaths.

I have no real idea. I was just saying this, as in, I don't think criminal hang around facebook.

If that'd be the case, would facebook just be some sort of crime catching tool ?

I wonder though, is sociopathy a crime, or some trait that will turn people into criminal ?

> I wouldn’t be surprised if some of the Facebook members who are most active and have the most ‘friends’ are sociopaths.

I doubt the people you are talking about can be criminal.

Crime and psychology, sigh

Re: Finding Sociopaths on Facebook

#20
post #2

An excellent post by Schneier. > The problem isn't just that such a system is wrong, it's that the mathematics of testing makes this sort of thing pretty ineffective in practice. It's called the "base rate fallacy." Suppose you have a test that's 90% accurate in identifying both sociopaths and non-sociopaths. If you assume that 4% of people are sociopaths, then the chance of someone who tests positive actually being…

This is a pretty bad example to illustrate the fallacy because a 25% confidence is actually extremely good. I don't know what good a test for sociopathy is, but if we had a test this good at identifying terrorists, it would be incredibly useful. If signals intelligence could produce a list of people and guarantee that a quarter of the people on that list are terrorists, it would absolutely revolutionize law enforceme…

This issue has nothing to do with the efficacy of the tests. In a perfect world having a list where 25% of the names are potential trouble-makers (for various definitions of trouble-maker) would be an enormous benefit and allow resources to be targeted more efficiently. The real problem is what happens in the imperfect world where law enforcement, government, self-imposed officers of authority get lazy or down-right malevolent. Do you really want to live in a world where your family members might be carted off, never to return, because their name came up on a list generated by a computer. Try asking the folks in North Korea, for example, whether this type of test is a good idea or not.
Post reply on HN