51–60 of 61 posts
Earlier quoted context omitted.
'it was much better when most stock market decisions were mostly based on "fundamentals"' I don't recall such a period. Is there a particular interval you're thinking of?
I'm actually not thinking about a specific period, I was just referring (maybe naively) to the time before computer assisted analysis became so widespread (before the eighties, I guess).
Every model is "wrong", by definition of it being a "model" and not "reality". It's one of the few mind opening things I've learnt at university. That's not a problem if you take it as an incentive to improve how much you know about the real world. It's a problem when you put the model before the people, and say that "models got us in trouble because of calibration problems". An economic crisis is not an unavoidable…
Earlier quoted context omitted.
Alternatively, it may be simple information theory: A model that takes in 100 bits of specification simply can not correctly describe a process that has 10,000 bit's worth of degrees of freedom. And that's before we talk about iteration over time, and before we get to the final killer you mention, which is when the models are ruined by their own application to the domain. I think radical underspecification is much mo…
Here's the thing: you're both right. It's both radically underspecified and overfitted. The information-theoretic argument demonstrate that a model cannot exactly match the reality unless it's as complex as the reality. This article speaks of the separate problem that economic models are not evaluated in any sort of experiments, and thus are prone to overfitting. This makes them unlikely to even approximate well. Con…
What if reality is self-similar at certain scales? You could generate something that resembles the whole from one part of it.
I think what the author is describing is simple overfitting. http://en.wikipedia.org/wiki/Overfitting It is quite a newbie mistake for a scientist to be surprised by it. It affects every kind of modelling. I thought maybe this article would talk about why economic models are worst than other kinds of models. There are issues that arise when applying scientific models to the economy caused by the fact that when even g…
Alternatively, it may be simple information theory: A model that takes in 100 bits of specification simply can not correctly describe a process that has 10,000 bit's worth of degrees of freedom. And that's before we talk about iteration over time, and before we get to the final killer you mention, which is when the models are ruined by their own application to the domain. I think radical underspecification is much mo…
If I'm correct, though, the OP is talking about creating a model with 100 bits of specification, and then creating a model of that model and trying to train those 100 bits, which seems like it should be a more tractable problem.
To me it sounds more like he's just rediscovered the fact that when you try to set a model's parameters based on a limited set of observations (he generated 3 years worth of data from his model, then trained parameters based on that data), there's a lot of uncertainty left over, and you won't necessarily get the right model.
This is quite obvious - if your observations only cover a limited portion of phase space, then you shouldn't be surprised that in a complex enough model multiple parameterizations will fit the observations equally well. You just didn't have enough freaking data to distinguish between the models! In all branches of science, we deal with this problem, and the solution is that you try to find the simplest possible model that accurately explains your data (or, as is happening in physics right now, you try to enumerate the next level of theories that reproduce current data so that you can figure out which experiments you'll need to run to distinguish between them).
So this has doesn't hint at any sort of fundamental flaw with modeling in general (and yeegads, it has even less to do with finance...) - it's just that he didn't have enough data to infer a proper parameterization. Don't build complex models and expect to train them on small datasets...
Earlier quoted context omitted.
Here's the thing: you're both right. It's both radically underspecified and overfitted. The information-theoretic argument demonstrate that a model cannot exactly match the reality unless it's as complex as the reality. This article speaks of the separate problem that economic models are not evaluated in any sort of experiments, and thus are prone to overfitting. This makes them unlikely to even approximate well. Con…
"The information-theoretic argument demonstrate that a model cannot exactly match the reality unless it's as complex as the reality." What if reality is self-similar at certain scales? You could generate something that resembles the whole from one part of it.
A great quote from George E P Box: All models are wrong. Some are useful.
Great discussion! The author doesn't seem to introduce the concept of training/testing datasets which absolutely critical to obtaining any reasonable model. So I don't buy the author's thesis that economic models are always wrong. The solution to the hypothetical problem posed in the article is to separate the historical dataset into training and testing groups. The models should be generated while only 'seeing' the…
"training/testing datasets which absolutely critical to obtaining any reasonable model" This is partly correct but, in general, too strong. Am I commenting on the OP? Not really! Why too strong? Because it assumes too little and sometimes more information is available and with the extra information a 'testing data set' may not be needed. Why are 'testing data sets' important? If about all you have to go on is the 'hi…
For instance a great part of growth in the last 100 years has been from man's ability to harness energy from fossil fuels. If your time line is narrow enough, you can disregard the point that fossil fuels is not unlimited, and project continued rise in extraction.
Another example is the baby boom, and the introduction of women into the paid work force which led to continued rise in property prices.
One more is the introduction of laws which suddenly compel people to invest in the stockmarket. It leads to short term asset inflation but generally makes worse investment all round.
That said, it is fitting that an economy is well modelled using the principles of hydraulics. See http://en.wikipedia.org/wiki/MONIAC_Computer