Finding Sociopaths on Facebook

An excellent post by Schneier.

> The problem isn't just that such a system is wrong, it's that the mathematics of testing makes this sort of thing pretty ineffective in practice. It's called the "base rate fallacy." Suppose you have a test that's 90% accurate in identifying both sociopaths and non-sociopaths. If you assume that 4% of people are sociopaths, then the chance of someone who tests positive actually being a sociopath is 26%. (For every thousand people tested, 90% of the 40 sociopaths will test positive, but so will 10% of the 960 non-sociopaths.) You have postulate a test with an amazing 99% accuracy -- only a 1% false positive rate -- even to have an 80% chance of someone testing positive actually being a sociopath.

Interestingly here he uses percentages to describe base rates and risk. Gerd Gigerenzer has a nice book, Reckoning with Risk, where he explains with many examples the problems of this approach. Gerd asks people to use real numbers instead, which are much easier to understand for most people.

Thus, Schneier's example becomes:

> Out of 1,000 people about 40 of will be sociopaths. You have a test that will tell you if someone is, or is not, a sociopath. The test will be correct 9 times out of 10. Bob has taken the test, and has been identified as a possible sociopath. The chance that Bob is actually a sociopath are actually about 1 in 4. This is because the test will tell you that 36 of the 40 sociopaths are sociopaths, but it will also incorrectly tell you that 96 non-sociopaths are sociopaths.

My writing is lousy, and other people will be able to clean this up, but even with my poor writing style it's easier for most people to follow and understand than the percentages.

This is alarmingly important when you're making a health decision - "Should I remove my breasts to reduce my risk of breast cancer?" for example.

(http://www.amazon.com/Reckoning-Risk-Learning-Live-Uncertain...)

EDIT: I use "sociopath" because it's in the source article. I agree with NNQ that it's very troubling to bandy around diagnostic labels like this, and deem people to be dangerous, just because of a tentative probabilistic diagnosis.

This is a pretty bad example to illustrate the fallacy because a 25% confidence is actually extremely good. I don't know what good a test for sociopathy is, but if we had a test this good at identifying terrorists, it would be incredibly useful. If signals intelligence could produce a list of people and guarantee that a quarter of the people on that list are terrorists, it would absolutely revolutionize law enforcement.

> and guarantee that a quarter of the people on that list are terrorists,

Honest question, what would happen to the other three quarters?

Agreed. And perhaps the notion of "dangerous sociopath" can be revised altogether. I'm no expert but in my understanding sociopathy is more about non empathic social behavior. It can be felt as creepy and depicted as violent in fiction, but that's doesn't make it a dangerous behavior in itself, and can actually be socially rewarded.

It can be argued that the lack of empathy can lead to a more violent behavior, but violent acts comming from deep personal relationship are plenty a dozen as well, who knows.

Bruce has a great point about how people overreact due to the base rate fallacy, but I'm afraid that it will forever be too subtle for legislators and judges.

My immediate thought is that if the base rate fallacy were part of their education, society would be better off, but legislators still have to play to their constituents and it's hard to have hope in that sphere.