Live data from Hacker News

Who You Are

nytimes.com

21–30 of 38 posts

Re: Who You Are

#21

It definitely makes a lot of sense. Just try speaking with someone about politics or religion. Even if you conclusively prove that the other persons views are not based in fact or reason they will refuse to acknowledge you are right and then usually get mad and stop talking to you. Humans are most certainly not 'rational beings'. Our thinking is constantly biased by our formative experiences and our environment.

Humans are most certainly not 'rational beings'. Our thinking is constantly biased by our formative experiences and our environment.

It goes far deeper than that, too. Our raw pattern matching sensitivity is cranked to the max at a very low level, and this hypersensitivity to perceived order ricochets throughout the entire system of data processing that our brain engages in.

We see patterns everywhere, whether they're real or not, and we have trouble unseeing them even once we know for a fact that the data is random, or that the pattern fails. Statistically speaking, we're a freaking mess, we're constantly pulled towards the wrong answers, we never have good estimates about how reliable our inferences are, it's just an all around bad scene.

And yet the combination of all of these seriously flawed pattern inferences leads to a creature that, all said and done, makes pretty damn useful predictions about a lot of things, even if the details of how those predictions get made are all wrong. This is surprising, since typically in statistics when we use algorithms that are too optimistic or sensitive we end up with pure garbage. If I had to guess, humans end up implementing something like the reverse of a typical boosting algorithm, in that we take a bunch of too-strong pattern recognizing subunits, and then put them together into something that pits them against each other to become more robust against mis-prediction, but I don't have any data to back up that assumption, or any clear idea how it might work - which is, I guess, a perfect example of exactly this kind of mental stupidity that we're so commonly driven by.

Re: Who You Are

#22

This article reminded me of my undergrad econ classes. I understand that it was just undergrad and we were learning a basic tool set. But it seriously scares me when I remember how my classmates and I (some of whom are on Wall St) took class after class that drilled supply/demand graphs premised on rational, utility maximizing populations into our heads. Whether the theories in this article are oversimplified or not,…

[deleted]

Re: Who You Are

#23

It definitely makes a lot of sense. Just try speaking with someone about politics or religion. Even if you conclusively prove that the other persons views are not based in fact or reason they will refuse to acknowledge you are right and then usually get mad and stop talking to you. Humans are most certainly not 'rational beings'. Our thinking is constantly biased by our formative experiences and our environment.

Humans are most certainly not 'rational beings'. Our thinking is constantly biased by our formative experiences and our environment. It goes far deeper than that, too. Our raw pattern matching sensitivity is cranked to the max at a very low level, and this hypersensitivity to perceived order ricochets throughout the entire system of data processing that our brain engages in. We see patterns everywhere , whether they'…

Makes me wonder if the singularity is going to be more about how AI handles random and non-random information than anything else.

Re: Who You Are

#24
post #19

Anyone else feel that the article was exceedingly lacking in subtance, especially for an NYTimes.com article?

I thought that the article was exceedingly lacking in substance, typically for an nytimes.com article.

Re: Who You Are

#25

It definitely makes a lot of sense. Just try speaking with someone about politics or religion. Even if you conclusively prove that the other persons views are not based in fact or reason they will refuse to acknowledge you are right and then usually get mad and stop talking to you. Humans are most certainly not 'rational beings'. Our thinking is constantly biased by our formative experiences and our environment.

Humans are most certainly not 'rational beings'. Our thinking is constantly biased by our formative experiences and our environment. It goes far deeper than that, too. Our raw pattern matching sensitivity is cranked to the max at a very low level, and this hypersensitivity to perceived order ricochets throughout the entire system of data processing that our brain engages in. We see patterns everywhere , whether they'…

"If I had to guess, humans end up implementing something like the reverse of a typical boosting algorithm, in that we take a bunch of too-strong pattern recognizing subunits, and then put them together into something that pits them against each other to become more robust against mis-prediction"

"Ensemble methods" seems to be what you're talking about. ( http://en.wikipedia.org/wiki/Ensemble_learning )

The application of many models put together to produce one signal to accurately predict the future.

I believe the Netflix challenge was won using ensemble methods acting in concert.

"Our final solution (RMSE=0.8712) consists of blending 107 individual results. Since many of these results are close variants, we first describe the main approaches behind them. "

( PDF paper: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.142... )

QIM, a large hedge fund that works futures, also uses the same model.

"In more direct language, Woodriff uses a statistical technique called the ensemble method, which is a way of mining data to produce something akin to the wisdom of crowds. A bundle of computer models, each searching for patterns in different ways, are linked together to produce a consensus statistical prediction—a sort of prediction by algorithmic committee. Scientists use the method to help predict ozone levels, for example. Woodriff uses it to help predict where futures markets are headed over a 24-hour period. His predictions are derived from four basic bits of historical pricing information: the open, close, high and low of specific markets.

Rishi Narang, whose Telesis Capital is a longtime investor in QIM, says other fund managers use similar methods and techniques. "The core idea is not so magical," Narang says. "It is how he puts it together. Getting the program correct is very challenging."

( http://www.absolutereturn-alpha.com/Article/2361672/QIMs-Jaf... )

Re: Who You Are

#26

Earlier quoted context omitted.

Humans are most certainly not 'rational beings'. Our thinking is constantly biased by our formative experiences and our environment. It goes far deeper than that, too. Our raw pattern matching sensitivity is cranked to the max at a very low level, and this hypersensitivity to perceived order ricochets throughout the entire system of data processing that our brain engages in. We see patterns everywhere , whether they'…

"If I had to guess, humans end up implementing something like the reverse of a typical boosting algorithm, in that we take a bunch of too-strong pattern recognizing subunits, and then put them together into something that pits them against each other to become more robust against mis-prediction" "Ensemble methods" seems to be what you're talking about. ( http://en.wikipedia.org/wiki/Ensemble_learning ) The applicatio…

[deleted]

Re: Who You Are

#27

Earlier quoted context omitted.

Humans are most certainly not 'rational beings'. Our thinking is constantly biased by our formative experiences and our environment. It goes far deeper than that, too. Our raw pattern matching sensitivity is cranked to the max at a very low level, and this hypersensitivity to perceived order ricochets throughout the entire system of data processing that our brain engages in. We see patterns everywhere , whether they'…

"If I had to guess, humans end up implementing something like the reverse of a typical boosting algorithm, in that we take a bunch of too-strong pattern recognizing subunits, and then put them together into something that pits them against each other to become more robust against mis-prediction" "Ensemble methods" seems to be what you're talking about. ( http://en.wikipedia.org/wiki/Ensemble_learning ) The applicatio…

Boosting, which he mentioned, is an ensemble method so I assume the parent is familiar with them.

Ensemble methods incorporate multiple weak classifiers and work to make them stronger. I think the parent was thinking of the reverse of this, although that idea seems pretty alien to me.

Re: Who You Are

#28
post #24
post #19

Anyone else feel that the article was exceedingly lacking in subtance, especially for an NYTimes.com article?

I thought that the article was exceedingly lacking in substance, typically for an nytimes.com article.

NYTimes makes some mistakes and publishes some junk, but they also have great content. It's been uneven. What's better?

I couldn't get my NYTimes this weekend, so being a Bay Area resident I got the SF Chronicle. The writing there is for grade school kids. Not trying to be funny, but it was sad.

(Don't get me started on the Economist...)

Re: Who You Are

#29
post #10

Earlier quoted context omitted.

What are the best books available on the subject of predicting human behavior, if you don't mind me asking? Advanced or technical are fine, scientific journals would be over my comprehension, I'm afraid.

One great book on behavioral economics that's based on a lot of research and experimental data is http://www.amazon.com/Judgment-Managerial-Decision-Making-Ba... Came highly recommended from a Stanford professor and is one of the most "meaty" books on the topic.

Animal Spirits is probably a lighter read (it's still on my to-read list), but I've been impressed when hearing the author, Robert Shiller, speak:

http://www.amazon.com/Animal-Spirits-Psychology-Economy-Capi...

Discussing animal spirits in various videos on youtube: http://www.youtube.com/results?search=Search&resnum=0&#3...

Re: Who You Are

#30

Earlier quoted context omitted.

"If I had to guess, humans end up implementing something like the reverse of a typical boosting algorithm, in that we take a bunch of too-strong pattern recognizing subunits, and then put them together into something that pits them against each other to become more robust against mis-prediction" "Ensemble methods" seems to be what you're talking about. ( http://en.wikipedia.org/wiki/Ensemble_learning ) The applicatio…

Boosting, which he mentioned, is an ensemble method so I assume the parent is familiar with them. Ensemble methods incorporate multiple weak classifiers and work to make them stronger. I think the parent was thinking of the reverse of this, although that idea seems pretty alien to me.

Indeed. The closest I can think of to what he is saying is pareto coevolution
Post reply on HN