Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The forecasting fallacy (2020)

alexmurrell.co.uk

Post reply on HN
Print view View original

21–30 of 71 posts

Re: The forecasting fallacy (2020)

#21

Post by civilized »

There's a germ of insight here that could use some development and nuancing.

> The future is uncertain. You cannot predict it. But you can create it.

For millions of years, prediction has been the engine by which humans have created the future. We don't always call it that, but prediction is the engine.

Let's start from the most basic facts of life. We know from experience (our own and others') what plants will sustain us and what will kill us, so we can predict what present-day choices of food will create a positive future. We create our positive future by making choices in accordance with those predictions.

It seems that we need to figure out what separates the kind of prediction that is the engine of human life and progress from the kind that is just useless blather.

Re: The forecasting fallacy (2020)

#22

Post by WJW »

chadash wrote: post #19
cesaref wrote: post #16

If you want a counter example, go and investigate algo trading hedge funds - you'll find they do a pretty solid job of predicting the future. Sure, some of them predict only a few ms into the future, some a few minutes (the one I worked for was in that category) and others will do interday strategies. I'm pretty sure there are examples which have a track record of decent returns above the markets they trade in with l…

I don’t have a background in this but I was under the impression that much of algorithmic trading is that there are trillions of pennies lying around and if you have an algorithm that picks up those pennies faster than anyone else, you make a lot of money. So it’s capitalizing on tiny market inefficiencies rather than directional predictions.

There's a wide variety of strategies available. The type you mention of picking up small inefficiencies certainly exists but there are plenty of other strategies that involve having some sort of informational edge. Some hedge fund managers just read a lot of earnings releases, but there are also more sophisticated approaches: a famous example would be the fund that paid for satellite imagery of the parking lots of certain shops, so that they could count how many cars there were and extrapolate that into whether the chain was growing or not.

Another straightforward example would involve using proprietary weather forecasting software to try and predict the global grain/cocoa/coffee/whatever harvest, so that you can then trade accordingly if you can see a bumper crop coming up.

Re: The forecasting fallacy (2020)

#23

Post by Zolde »

hef19898 wrote: post #17
Zolde wrote: post #10

One should ask economists what a recession is, not how to predict one. Good modelers do not necessarily need (or want) to know what they are predicting and still beat "domain experts". Authority without clear track-record is a net negative to getting good results. It is better to stick to anonymity, and only let the track-record do the talking/weighting. Without a clear track-record it does not even matter if the pre…

Consultancies predicting something isn't forecasting, it is marketing. And there or only a rare few thing I disagree more stongly with the statement, that good modellers / data scientist / whatever only need knowledge about how to model stuff to beat domain experts. It takes domain experts to judge whether or not a model correct, to identify the known and unknown unknowns and limitations of these models. Claiming oth…

"Every time I fire a linguist, the performance of the speech recognizer goes up."

> It takes domain experts to judge whether or not a model correct, to identify the known and unknown unknowns and limitations of these models.

Arguably true, but I still claim the domain expert test-performance is below that of a modeling expert. No knowledge/preconceptions: Try it all, let evaluation decide. Expert domain knowledge/preconceptions: This can't possibly work!

Domain experts need to focus on decision science (what policies to build on top of model output). Data scientists need to focus on providing model output to make the most accurate/informed decisions downstream.

Re: The forecasting fallacy (2020)

#24

Post by altdataseller »

WJW wrote: post #22
chadash wrote: post #19

Earlier quoted context omitted.

I don’t have a background in this but I was under the impression that much of algorithmic trading is that there are trillions of pennies lying around and if you have an algorithm that picks up those pennies faster than anyone else, you make a lot of money. So it’s capitalizing on tiny market inefficiencies rather than directional predictions.

There's a wide variety of strategies available. The type you mention of picking up small inefficiencies certainly exists but there are plenty of other strategies that involve having some sort of informational edge. Some hedge fund managers just read a lot of earnings releases, but there are also more sophisticated approaches: a famous example would be the fund that paid for satellite imagery of the parking lots of ce…

Both of those examples have been exploited to death and no longer are profitable

Re: The forecasting fallacy (2020)

#25

Post by cnewey »

altdataseller wrote: post #24
WJW wrote: post #22

Earlier quoted context omitted.

There's a wide variety of strategies available. The type you mention of picking up small inefficiencies certainly exists but there are plenty of other strategies that involve having some sort of informational edge. Some hedge fund managers just read a lot of earnings releases, but there are also more sophisticated approaches: a famous example would be the fund that paid for satellite imagery of the parking lots of ce…

Both of those examples have been exploited to death and no longer are profitable

If the parent had discovered a viable and profitable trading strategy, do you think they would share it here?

Re: The forecasting fallacy (2020)

#26

Post by WJW »

altdataseller wrote: post #24
WJW wrote: post #22

Earlier quoted context omitted.

There's a wide variety of strategies available. The type you mention of picking up small inefficiencies certainly exists but there are plenty of other strategies that involve having some sort of informational edge. Some hedge fund managers just read a lot of earnings releases, but there are also more sophisticated approaches: a famous example would be the fund that paid for satellite imagery of the parking lots of ce…

Both of those examples have been exploited to death and no longer are profitable

True, but they were just meant as easy examples of non-HFT hedge fund strategies.

Re: The forecasting fallacy (2020)

#27

Post by troupe »

jxf wrote: post #14

The author would also conclude: * Collision avoidance systems are terrible at forecasting collisions because they almost never result in a collision. (The point of the system is to help you avoid an upcoming collision.) * The prediction that Y2K would happen was a bad one since it didn't happen. (We spent billions of dollars to make sure it didn't.) * The 1978 prediction that the ozone layer would be depleted by 2010…

> The author would also conclude: > The prediction that Y2K would happen was a bad one since it didn't happen. (We spent billions of dollars to make sure it didn't.)

I would argue that we don't really know what would have happened had the world not spent all the money on upgrading systems. It appears a very large number of them would have continued to work as expected and it isn't immediately clear if the ones that were replaced would have resulted in a catastrophe.

Re: The forecasting fallacy (2020)

#28

Post by hef19898 »

Zolde wrote: post #23
hef19898 wrote: post #17

Earlier quoted context omitted.

Consultancies predicting something isn't forecasting, it is marketing. And there or only a rare few thing I disagree more stongly with the statement, that good modellers / data scientist / whatever only need knowledge about how to model stuff to beat domain experts. It takes domain experts to judge whether or not a model correct, to identify the known and unknown unknowns and limitations of these models. Claiming oth…

"Every time I fire a linguist, the performance of the speech recognizer goes up." > It takes domain experts to judge whether or not a model correct, to identify the known and unknown unknowns and limitations of these models. Arguably true, but I still claim the domain expert test-performance is below that of a modeling expert. No knowledge/preconceptions: Try it all, let evaluation decide. Expert domain knowledge/pre…

I'll be blunt: everytime I saw people try model something they don't understand, it boiled down to throwing stuff at the wall and see what sticks. Very best case, whatever stuck solved one special case without people realizing it was a speciap case.

Worst case, the stuff sticking was sheer luck, could have, and quite often was, identified prior of trying by domain experts, no lessons were drawn from the excercise and the resulting models were ignored by everyone except the modellers.

Re: The forecasting fallacy (2020)

#29

Post by hamilyon2 »

Isn't inability to accurately predict some economic metrics consequence of efficient market hypothesis?

All available and some unavailable information is already reflected in market. So, sum of reasonable guesses of next year GDP more or less is today's market index. Anything over that is some baseless speculation with no skin in the game.

Re: The forecasting fallacy (2020)

#30

Post by nimbius »

hamilyon2 wrote: post #29

Isn't inability to accurately predict some economic metrics consequence of efficient market hypothesis? All available and some unavailable information is already reflected in market. So, sum of reasonable guesses of next year GDP more or less is today's market index. Anything over that is some baseless speculation with no skin in the game.

correct me if im wrong but the approximate cycle of boom/bust each decade or so for capitalism is a well documented feature? that it sort of has to "reinvent" itself each time in order for continued existence?

couldnt one plan around this in broader strokes that dont involve the sorts of precision quantitative analysis that wallstreet seems so fond of?

Post reply on HN