Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Predict the future with Machine Learning

zeroequalsfalse.press

Post reply on HN
Print view View original

41–46 of 46 posts

Re: Predict the future with Machine Learning

#41

Post by sgt101 »

ryanwaggoner wrote: post #39
calafrax wrote: post #37

Earlier quoted context omitted.

is that a joke? weather predictions are notoriously unreliable even though they are given with extreme granularity. that aside you are missing a larger point. if you predict the future based on past data all you are saying is "the future will be the same as the past." you aren't predicting anything. you will be wrong every single time something novel occurs, which is pretty frequently in the real world.

The perception that weather forecasting is notoriously unreliable is mostly false: https://mobile.nytimes.com/2012/09/09/magazine/the-weatherma...

From your link : "Why are weather forecasters succeeding when other predictors fail? It’s because long ago they came to accept the imperfections in their knowledge. That helped them understand that even the most sophisticated computers, combing through seemingly limitless data, are painfully ill equipped to predict something as dynamic as weather all by themselves. So as fields like economics began relying more on Big Data, meteorologists recognized that data on its own isn’t enough."

Re: Predict the future with Machine Learning

#42

Post by danieltillett »

taneq wrote: post #38
ben_w wrote: post #10

Earlier quoted context omitted.

Black Swans are the error rate of your predictions (the real error rather than your prediction of your error rate) not existential proof that prediction is always doomed. After all, if Black Swans were common enough to make prediction a fool's errand most of the time, the bird of that name would never have led to the book of that name, because everyone would be predicting their failure to predict things .

I always find the term 'black swan' to be interesting, because where I live, black swans are the rule rather than the exception. I think this just makes the analogy even better, since it highlights how much your ability to predict events depends on your environment.

Me too :) It is not a term that get used much here (Australia).

I have to say I rather prefer the black variety over the white.

Re: Predict the future with Machine Learning

#43

Post by ben_w »

sgt101 wrote: post #34
ben_w wrote: post #10

Earlier quoted context omitted.

Black Swans are the error rate of your predictions (the real error rather than your prediction of your error rate) not existential proof that prediction is always doomed. After all, if Black Swans were common enough to make prediction a fool's errand most of the time, the bird of that name would never have led to the book of that name, because everyone would be predicting their failure to predict things .

I think that a Black Swan is when a new factor appears in your domain. In science we are conditioned from the start to create fair tests in controlled experiments. Control is the fundamental of experiment - and statistics are designed to handle experimental data. In the real world there are often no controls, and complex systems can be driven by an attractor for a very long time before one morning they are not, and e…

That's certainly one type of Black Swan. Taleb's example of that sort of thing being a Turkey predicting they will be fed (because that is what happened every other day of their life) but who is actually slaughtered.

However, it is not the only type. There is also the stock market, which demonstrates major unpredictability every few years, but which can also be approximated the same way between each of the Black Swans. (And they keep being Black Swans because the gap between them is large enough for people to convince themselves that "This time it's different, this time n̵o̵b̵o̵d̵y̵ ̵w̵i̵l̵l̵ ̵h̵a̵v̵e̵ ̵t̵o̵ ̵b̵e̵ ̵n̵a̵i̵l̵e̵d̵ ̵t̵o̵ ̵a̵n̵y̵t̵h̵i̵n̵g̵ growth will be eternal!")

Edit:

Point is, it generalises as how wrong you are in your predictions, and the closer your estimate of your error rate is to your actual error rate, the better your model is.

Re: Predict the future with Machine Learning

#44

Post by calafrax »

ryanwaggoner wrote: post #39
calafrax wrote: post #37

Earlier quoted context omitted.

is that a joke? weather predictions are notoriously unreliable even though they are given with extreme granularity. that aside you are missing a larger point. if you predict the future based on past data all you are saying is "the future will be the same as the past." you aren't predicting anything. you will be wrong every single time something novel occurs, which is pretty frequently in the real world.

The perception that weather forecasting is notoriously unreliable is mostly false: https://mobile.nytimes.com/2012/09/09/magazine/the-weatherma...

i am not sure what that article is supposed to prove. it doesn't contain any study results on the accuracy of meteorological predictions.

I don't have the data handy but to the best of my recollection weather forecasting for high/low temperature and precipitation does pretty well for the range of 24-48 hours but declines steadily in accuracy, and is no better than random guess around 2 weeks out.

That said, you are not addressing my other point, which is that "weather prediction" is just saying "things are going to stay the same." You are always starting with a set of conditions and then looking at your records and seeing what happened in similar conditions and predicting that the same thing will happen again.

Predicting that things will stay the same may come out as better than random guess in many cases but it will still be 100% wrong in cases where something novel happens.

Re: Predict the future with Machine Learning

#45

Post by gbrown »

sgt101 wrote: post #35
gbrown wrote: post #14

Earlier quoted context omitted.

What??? Prediction is certainly a type of extrapolation, but to claim that it's "mathematically invalid" reveals a severe lack of knowledge on your part. In fact, under parametric assumptions about the data generating mechanism, we can exactly quantify the expected coverage of prediction intervals. That's literally a standard topic in an introductory statistics course.

Hello, I think Calafrax is probably right. :o) I think you implicitly agree because you say "under parametric assumptions..." which means you know whats going on; but to make the point-> Statistics as we know it "works" (can be derived) under the assumptions of controlled experimental data. As a thought experiment think about the weather - we know that if we build a classifier that predicts the weather in my garden t…

The point is, statistical prediction is definitely a thing, and is not "mathematically invalid" - it's mathematically well defined, with predictable consequences (increasing variance as the extrapolation becomes greater). Certainly, statistical models are not Crystal balls, but they never claimed to be. If you have a reasonable frequentist model and good data about an ongoing process, you should be able to make predictions with reasonable confidence bounds. If you have a reasonable Bayesian model and good data about an ongoing process, you should be able to coherently quantify your uncertainty about the future state of the system.

Obviously, this is more or less feasible in practice, depending on the phenomenon under study. Calling markets unpredictable is not evidence against the existence of rigorous frameworks for statistical prediction.

Don't let bad experiences with inexperienced and overconfident practitioners blind you to established, uncontroversial, mathematical truths.

Re: Predict the future with Machine Learning

#46

Post by gbrown »

sgt101 wrote: post #41
ryanwaggoner wrote: post #39

Earlier quoted context omitted.

The perception that weather forecasting is notoriously unreliable is mostly false: https://mobile.nytimes.com/2012/09/09/magazine/the-weatherma...

From your link : "Why are weather forecasters succeeding when other predictors fail? It’s because long ago they came to accept the imperfections in their knowledge. That helped them understand that even the most sophisticated computers, combing through seemingly limitless data, are painfully ill equipped to predict something as dynamic as weather all by themselves. So as fields like economics began relying more on Bi…

Quantifying uncertainty is one of the main points of statistics. Don't confuse the limitations of point estimates provided by machine learning techniques with all of statistical practice.
Post reply on HN