Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Predict the future with Machine Learning

zeroequalsfalse.press

Post reply on HN
Print view View original

31–40 of 46 posts

Re: Predict the future with Machine Learning

#31

Post by srean »

thousandpounds wrote: post #17
hal9000xp wrote: post #2

I would like to read mathematical foundations of machine learning written for those who are bad at calculus but good at discrete mathematics and algorithms. For example, I'm learning algorithms, participate at contests, quite comfortable with combinatorics and discrete probability theory but I'm absolute zero at calculus. I would like to read machine learning's math introduction which is friendly to my "discrete" bra…

I am in the same boat. I get the feeling that most Calculus books are just a compilation of tips and tricks. So I am suggesting you invest time into learning real analysis proper. Right now I am learning from [1]. It follows Rudin closely and as opposed to many other analysis books meant to "better explain" stuff, it goes deep into the trenches and actually tackles the subject. [1] https://www.amazon.com/Real-Analysi…

By most books I assume you mean pretty college US text books. I have no experience with them, are they really that bad ?

My previous college calculus was far from the rigour of Rudin but also far from a cookbook flavour. Unless by 'cookbook' you mean the chain rule and differentials of standard forms. It's not that hard to teach this stuff at least it's no harder than, say, geometry. I found trigonometry far more difficult.

We were first taught limits then came differentiation of polynomials using infitesimals.Chain rule was introduced. Then we ventured into differentiation of other functions. Integration was first introduced much like riemann integrals then came integration as an inverse of differentiation.

Re: Predict the future with Machine Learning

#33

Post by hprotagonist »

shincert wrote: post #15
hprotagonist wrote: post #13

Fundamentally it is Software that works like our brain.. Stopped here. Moving on.

Why? Is it really that bad of an analogy for an absolute beginner?

I think so. Primarily due to Djikstra's anti-anthropomorphic stance, which is very important here.

1. as the other poster to you noted, people are more apt to conflate "strong AI" with what we're actually doing with tensorflow, leading to very weird overreactions that aren't germane.

2. just as importantly, developers who believe this line of thinking are biased against a more correct understanding of their code, which makes debugging much more difficult and prevents advances in the underlying technology.

The implied abstraction ... is however beyond the computing scientist imbued with the operational approach that the anthropomorphic metaphor induces. In a very real and tragic sense he has a mental block: his anthropomorphic thinking erects an insurmountable barrier between him and the only effective way in which his work can be done well.

Re: Predict the future with Machine Learning

#34

Post by sgt101 »

ben_w wrote: post #10
tinkerdol wrote: post #8

We shouldn't ever confuse machine learning with predicting the future -- just because you've never encountered a black swan in the wild, doesn't mean they don't exist. That being said, the article otherwise seems like a great introduction. Not sure why they chose that title.

Black Swans are the error rate of your predictions (the real error rather than your prediction of your error rate) not existential proof that prediction is always doomed. After all, if Black Swans were common enough to make prediction a fool's errand most of the time, the bird of that name would never have led to the book of that name, because everyone would be predicting their failure to predict things .

I think that a Black Swan is when a new factor appears in your domain. In science we are conditioned from the start to create fair tests in controlled experiments. Control is the fundamental of experiment - and statistics are designed to handle experimental data.

In the real world there are often no controls, and complex systems can be driven by an attractor for a very long time before one morning they are not, and every rule that you have is useless (often worse than useless).

Sources of error are not equal; "Black Swan Error" is unusual in that over time it may be that this source is more important than any other source of data in your domain - the strange attractor that drove the creation of your classifier over the last 20 years may never recapture your function and if that's the case your classifier will be literally the most wrong thing you could have!

Re: Predict the future with Machine Learning

#35

Post by sgt101 »

gbrown wrote: post #14
calafrax wrote: post #9

Earlier quoted context omitted.

It is an unfortunate misconception that statistical probability can be used to predict the future. Any time you extend a statistical model temporally it immediately becomes mathematically invalid since probabilistic statistics are only valid for a fixed population at a fixed moment in time. Unfortunately business and government is rife with people predicting the future based on statistical models that have no more ma…

What??? Prediction is certainly a type of extrapolation, but to claim that it's "mathematically invalid" reveals a severe lack of knowledge on your part. In fact, under parametric assumptions about the data generating mechanism, we can exactly quantify the expected coverage of prediction intervals. That's literally a standard topic in an introductory statistics course.

Hello, I think Calafrax is probably right. :o) I think you implicitly agree because you say "under parametric assumptions..." which means you know whats going on; but to make the point->

Statistics as we know it "works" (can be derived) under the assumptions of controlled experimental data. As a thought experiment think about the weather - we know that if we build a classifier that predicts the weather in my garden tomorrow based on the history of the weather in my garden it will do very badly. Why - well because weather is very very very complex; the range of behavior is vast. But worse, it's unstable. The weather in my garden is driven by several complex systems; the ocean, the atmosphere, the earth's orbit and sol! Statistics can't predict the future of the weather in my garden.

Statistics also can't predict other things like the future of the financial markets (not least because if you find a statistical law about that they you will act on it and then screw it up)

It's important to me to bang on about this because there are loads of people who sit through their introductory courses and read the example of predicting a biased roulette wheel. Years later they end up running the company/country/community that I live in and they have a view that they can use the same principles to do it... and this thinking leads to nasty surprises for me.

Re: Predict the future with Machine Learning

#36

Post by panarky »

sgt101 wrote: post #35
gbrown wrote: post #14

Earlier quoted context omitted.

What??? Prediction is certainly a type of extrapolation, but to claim that it's "mathematically invalid" reveals a severe lack of knowledge on your part. In fact, under parametric assumptions about the data generating mechanism, we can exactly quantify the expected coverage of prediction intervals. That's literally a standard topic in an introductory statistics course.

Hello, I think Calafrax is probably right. :o) I think you implicitly agree because you say "under parametric assumptions..." which means you know whats going on; but to make the point-> Statistics as we know it "works" (can be derived) under the assumptions of controlled experimental data. As a thought experiment think about the weather - we know that if we build a classifier that predicts the weather in my garden t…

> if we build a classifier that predicts the weather in my garden tomorrow based on the history of the weather in my garden it will do very badly

Give me hourly readings of temperature, wind speed, wind direction, precipitation, cloud cover and barometric pressure for the last 10 years and I can give you a very accurate prediction of tomorrow's weather in your garden.

Re: Predict the future with Machine Learning

#37

Post by calafrax »

panarky wrote: post #36
sgt101 wrote: post #35

Earlier quoted context omitted.

Hello, I think Calafrax is probably right. :o) I think you implicitly agree because you say "under parametric assumptions..." which means you know whats going on; but to make the point-> Statistics as we know it "works" (can be derived) under the assumptions of controlled experimental data. As a thought experiment think about the weather - we know that if we build a classifier that predicts the weather in my garden t…

> if we build a classifier that predicts the weather in my garden tomorrow based on the history of the weather in my garden it will do very badly Give me hourly readings of temperature, wind speed, wind direction, precipitation, cloud cover and barometric pressure for the last 10 years and I can give you a very accurate prediction of tomorrow's weather in your garden.

is that a joke? weather predictions are notoriously unreliable even though they are given with extreme granularity.

that aside you are missing a larger point. if you predict the future based on past data all you are saying is "the future will be the same as the past." you aren't predicting anything. you will be wrong every single time something novel occurs, which is pretty frequently in the real world.

Re: Predict the future with Machine Learning

#38

Post by taneq »

ben_w wrote: post #10
tinkerdol wrote: post #8

We shouldn't ever confuse machine learning with predicting the future -- just because you've never encountered a black swan in the wild, doesn't mean they don't exist. That being said, the article otherwise seems like a great introduction. Not sure why they chose that title.

Black Swans are the error rate of your predictions (the real error rather than your prediction of your error rate) not existential proof that prediction is always doomed. After all, if Black Swans were common enough to make prediction a fool's errand most of the time, the bird of that name would never have led to the book of that name, because everyone would be predicting their failure to predict things .

I always find the term 'black swan' to be interesting, because where I live, black swans are the rule rather than the exception. I think this just makes the analogy even better, since it highlights how much your ability to predict events depends on your environment.

Re: Predict the future with Machine Learning

#39

Post by ryanwaggoner »

calafrax wrote: post #37
panarky wrote: post #36

Earlier quoted context omitted.

> if we build a classifier that predicts the weather in my garden tomorrow based on the history of the weather in my garden it will do very badly Give me hourly readings of temperature, wind speed, wind direction, precipitation, cloud cover and barometric pressure for the last 10 years and I can give you a very accurate prediction of tomorrow's weather in your garden.

is that a joke? weather predictions are notoriously unreliable even though they are given with extreme granularity. that aside you are missing a larger point. if you predict the future based on past data all you are saying is "the future will be the same as the past." you aren't predicting anything. you will be wrong every single time something novel occurs, which is pretty frequently in the real world.

The perception that weather forecasting is notoriously unreliable is mostly false: https://mobile.nytimes.com/2012/09/09/magazine/the-weatherma...

Re: Predict the future with Machine Learning

#40

Post by sgt101 »

panarky wrote: post #36
sgt101 wrote: post #35

Earlier quoted context omitted.

Hello, I think Calafrax is probably right. :o) I think you implicitly agree because you say "under parametric assumptions..." which means you know whats going on; but to make the point-> Statistics as we know it "works" (can be derived) under the assumptions of controlled experimental data. As a thought experiment think about the weather - we know that if we build a classifier that predicts the weather in my garden t…

> if we build a classifier that predicts the weather in my garden tomorrow based on the history of the weather in my garden it will do very badly Give me hourly readings of temperature, wind speed, wind direction, precipitation, cloud cover and barometric pressure for the last 10 years and I can give you a very accurate prediction of tomorrow's weather in your garden.

Hello, interestingly governments and private industries have invested a very large amount of money in the launch of satellites, development of supercomputers and code and the training of forecasters to interpret them.

Many years ago I actually seriously tried to do what you describe above, I tried out all sorts of things around seasonal analysis and other features. What kills it is the chaotic nature of UK weather due to the jetstream and NAO.

Post reply on HN