This was pretty much proven for stocks too
Why Do Models That Predict Failure Fail? [pdf]
21–29 of 29 posts
Re: Why Do Models That Predict Failure Fail? [pdf]
#22Re: Why Do Models That Predict Failure Fail? [pdf]
#23Earlier quoted context omitted.
Excerpted summary from here [0] Difficulties of Prediction Ultimately, our world and future are unknowable and unpredictable because of various factors, including: • The role of chance and luck in inventions and game-changing discoveries • The inherently unpredictable nature of dynamical systems and variables that behave in inconsistent or nonlinear ways • We can’t make accurate predictions with flawed and incomplete…
These are characteristic talebian points (it's good that you read a Cliff's Notes, I read the whole thing and what a pompous gasbag) but there isn't much that's specifically talebian to what you said. People make predictions because they judge that it's better to have bad predictions than none at all. The probable error in prediction (e.g. the "cone of uncertainty" that becomes wider and wider as you go into the futu…
At the current velocity, there will be a catastrophic event to the driver. However, what do people normally do in these situations? With proper prudence, they come to a safe stop before the wall. Some will, at the last moment slam the brakes and come to a sliding, screeching stop before hitting the wall. Another set will swerve and wrap around a tree or end up in a ditch. While a very small amount will actually never hit their brakes and hit the wall at said velocity.
The initial data statement regarding velocity is technically not wrong, at all. Its extremely logical and true. However it lacks taking into account the human factor. Humans are not equations. All prediction methods regarding human action revolve around math, loosely or strongly. We dont possess math that accurately depicts human actions. Folks like Taleb are so far removed from having social skills or friends, he forgets other humans exist. Perhaps it's a sociopathic or psychopathic tendency where they see no value in other humans. I always forget which one is which.
Now, we can argue economics is a study to do this in a narrow field. Which I agree to. However, the best educated economists who aren't trying to sell you a get rich quick scheme all say roughly the same thing, "people are cray cray".
Poker is also a good example of this logic. Particularly the meta play the experts are at. Poker is beyond the standard statistical average of hands, which is all there is to blackjack. You still have the human element. That's what makes it more difficult.
People like Taleb try to over simplify concepts that scare them because they dont like to be wrong. Tetlock proved in various gov studies that people can become substantially BETTER (obviously not perfect nor does he claim to have a perfect solution) than average guessing by embracing the chance of being wrong. When embracing the potential of being wrong, you list out the potentials, the whys and then figure out the catalysts. Instead of having a single cone of prediction that widens over time, you have multiple smaller ones based on potential future actions. You apply confidence rates to each one as part of your prediction process.
I... dont remember why I got on that rant. I remember I wanted to make a joke that Taleb is an idiot and essentially agree with you, but wanted to give a good reason why along with it. Moral of the story, forecasting the near future is possible, but takes a lot of work and research to do it effectively. Dynamic and complex systems are not impossible to understand if you're willing to put in the work and in doing so, you will only be "so wrong" rather than blatantly wrong... second moral Taleb is a twat.
Re: Why Do Models That Predict Failure Fail? [pdf]
#24Earlier quoted context omitted.
Excerpted summary from here [0] Difficulties of Prediction Ultimately, our world and future are unknowable and unpredictable because of various factors, including: • The role of chance and luck in inventions and game-changing discoveries • The inherently unpredictable nature of dynamical systems and variables that behave in inconsistent or nonlinear ways • We can’t make accurate predictions with flawed and incomplete…
These are characteristic talebian points (it's good that you read a Cliff's Notes, I read the whole thing and what a pompous gasbag) but there isn't much that's specifically talebian to what you said. People make predictions because they judge that it's better to have bad predictions than none at all. The probable error in prediction (e.g. the "cone of uncertainty" that becomes wider and wider as you go into the futu…
Some do. (Some make predictions because they mistakenly believe that they can predict accurately.)
But is a bad prediction better than none at all? Depends on how much you rely on it. The more you rely on it, the more it matters that it is accurate.
Re: Why Do Models That Predict Failure Fail? [pdf]
#25A true future predicting algorithm should be aware of its existence and the affect its prediction will have over the receiver / receivers.
Re: Why Do Models That Predict Failure Fail? [pdf]
#26A true future predicting algorithm should be aware of its existence and the affect its prediction will have over the receiver / receivers.
Re: Why Do Models That Predict Failure Fail? [pdf]
#27A true future predicting algorithm should be aware of its existence and the affect its prediction will have over the receiver / receivers.
Consider the problem of predicting an agent's binary choice. If an agent follows a strategy of always choosing the opposite of what the algorithm predicts, then the algorithm's prediction will always be incorrect. A "true future predicting algorithm" cannot correctly predict the actions of certain kinds of agents, as long as those agents can observe the prediction.
Re: Why Do Models That Predict Failure Fail? [pdf]
#28> Using data from the Panel Study of Income Dynamics reveals that the relationship between predictors typically used to predict mortgage performance and the subsequent arrival of mortgage default trigger events has exhibited significant intertemporal heterogeneity. A pretty roundabout way of saying that the future was being predicted by extrapolating the past, and that the actual future was unlike the past. > Interes…
"intertemporal heterogeneity in the relationship between variables that are frequently used to predict [...] imply that model instability is a significant source of risk for [folks] that rely heavily on predictive statistical models and machine learning algorithms [...]"
No shit, Sherlock! aka "overfitting"
I don't mean to deride the actual research that was done, but I would hope that people understand this general property better before jumping on the bandwagon to use statistical models -- without needing the benefit of hindsight.
Re: Why Do Models That Predict Failure Fail? [pdf]
#29Earlier quoted context omitted.
These are characteristic talebian points (it's good that you read a Cliff's Notes, I read the whole thing and what a pompous gasbag) but there isn't much that's specifically talebian to what you said. People make predictions because they judge that it's better to have bad predictions than none at all. The probable error in prediction (e.g. the "cone of uncertainty" that becomes wider and wider as you go into the futu…
> People make predictions because they judge that it's better to have bad predictions than none at all. Some do. (Some make predictions because they mistakenly believe that they can predict accurately.) But is a bad prediction better than none at all? Depends on how much you rely on it. The more you rely on it, the more it matters that it is accurate.
Sometimes (in a iterative process, for example) all you need is a starting point. But some iterative processes can get stuck in infinite loops with a bad value. So sometimes a bad prediction would be better than none at all because you just need a starting point. But sometimes a bad prediction would be worse than none at all because you'd be in a non-halting process.