I'm currently learning machine learning at the most basic level, this is the sort of stuff I want to work towards though I deal with time series data a lot at work, I work in broadcasting/media and 99% of the time the data is fairly "predictable" and follows a regular daily pattern, peppered with the odd spikes during big, unpredicatble news events.
Time series analysis requires the data to be stationary.
Time Series Prediction Using LSTM Deep Neural Networks
51–60 of 72 posts
Re: Time Series Prediction Using LSTM Deep Neural Networks
#52I find it interesting that Computer Scientists are basically rediscovering statistics. Now when predicting time series, an issue is that most model (like ARIMA, GARCH etc.) are short-memory processes. When you look at the full-series prediction of LSTMs, you observe the same thing. So in terms of Time Series, Machine Learning is currently in the mid to late 80's compared to Financial Econometrics. So if you are a CS,…
It's been amazing to watch CS (really the Python community, save statsmodels and patsy) discover statistics. For a while I thought perhaps it was me and statistics that was "behind." Over time I realized that it was mostly re-invention of old ideas: one-hot encoding = dummy variables, neural networks approximating polynomial regression, etc. I decided to double-down on statistics and it's really paid off. NN / random…
The Phineas Gage of applied quantitative Econ is demand estimation. You typically want to know the elasticity of quantities sold to price so to inform pricing policies. But the problem is that causality is cloudy -- low prices cause a decrease in supply -- so you never know what you're looking at.
People with a decent training in econometrics know how to treat this problem.
I'm pretty sure orgs like Amazon were trying to do naive demand estimation, fell flat on their noses and copped to having to hire people who have thought about the underlying conceptual issues before.
Re: Time Series Prediction Using LSTM Deep Neural Networks
#53Earlier quoted context omitted.
It's been amazing to watch CS (really the Python community, save statsmodels and patsy) discover statistics. For a while I thought perhaps it was me and statistics that was "behind." Over time I realized that it was mostly re-invention of old ideas: one-hot encoding = dummy variables, neural networks approximating polynomial regression, etc. I decided to double-down on statistics and it's really paid off. NN / random…
Because saying "relevant username" is frowned upon I'll just point out that R A Fisher is "a genius who almost single-handedly created the foundations for modern statistical science"[0] 0. https://en.m.wikipedia.org/wiki/Ronald_Fisher
Those are criticism on personality, on the technical side it took the community a long time to undo the damage of promoting non robust parametric statistics. But this much is certain he pulled statistics into the realms of math -- no mean feat.
Re: Time Series Prediction Using LSTM Deep Neural Networks
#54I find it interesting that Computer Scientists are basically rediscovering statistics. Now when predicting time series, an issue is that most model (like ARIMA, GARCH etc.) are short-memory processes. When you look at the full-series prediction of LSTMs, you observe the same thing. So in terms of Time Series, Machine Learning is currently in the mid to late 80's compared to Financial Econometrics. So if you are a CS,…
I don't think it's entirely fair to say "Computer Scientists are basically rediscovering statistics". LSTMs are used beyond just time series prediction. It is also quite common in language modelling tasks, which is also a sequence modelling task, and where it works quite well. I'm not familiar at all with using GARCH/ARIMA for something like this. Also, with neural networks it's very easy and natural to build complex…
That would be nonparametric statistics.
Re: Time Series Prediction Using LSTM Deep Neural Networks
#55Earlier quoted context omitted.
exactly, Judea Pearl's The Book of Why opened my eyes to the fact that most of what happens in machine learning is really just curve fitting It connected with what i've heard Chomsky say about trying to develop laws of physics by filming what's happening outside the window. We need to do experiments and interventions to learn the dynamics of a system "What do you think the role is, if any, of other uses of so-called…
I really violently oppose this characterization of ML as "just" curve fitting, as if curve fitting is some simple solved problem. It seems like there is a ignorance about issues relating to model selection, which is an essential part of curve fitting. What complexity of model does the data support? Can you keep a distribution over structures that allows uncertain parts of the model to be interrogated? These are the p…
What you are saying is curve fitting with good predictive ability is not trivial, and that is indeed true.
Re: Time Series Prediction Using LSTM Deep Neural Networks
#56Earlier quoted context omitted.
It's been amazing to watch CS (really the Python community, save statsmodels and patsy) discover statistics. For a while I thought perhaps it was me and statistics that was "behind." Over time I realized that it was mostly re-invention of old ideas: one-hot encoding = dummy variables, neural networks approximating polynomial regression, etc. I decided to double-down on statistics and it's really paid off. NN / random…
Because saying "relevant username" is frowned upon I'll just point out that R A Fisher is "a genius who almost single-handedly created the foundations for modern statistical science"[0] 0. https://en.m.wikipedia.org/wiki/Ronald_Fisher
All the theory behind t-testing, Wald testing, using the detivatives of the log likelihood near to the MLE point estimate in order to also estimate standard errors when no analytical solution exists, ANOVA, instrumental variables, etc.
It is in no sense exaggerative or incendiary to say that whole collection of stuff is truly garbage statistics that is insanely rife with counter-intuitive results, common situations when minor violations of the assumptions can easily lead to statistically significant results of the wrong sign, and common practical needs (like model selection without doing a bunch of pairwise or subset selection calculations, or correcting for multicollinearity in large regressions where calculating something like variance inflation factors is totally intractable) are difficult or impossible.
Modern Bayesian approaches fully and entirely subsume these techniques, and not just for large data (in fact, using Bayesian methods is more critical for small data), and also not because of modern computing frameworks, but because, from very first principle of null-hypothesis significance testing, that whole field of stats/econometrics is fundamentally incapable of giving evidence or estimations that could address the very questions that the whole field is based on.
NHST basically solves a type of inference problem that nobody can ever actually have in reality, and which is almost always not even approximately close enough to actually be non-misleading.
NHST is like the stats analogue of Javascript: a horrible historical accident that gained market traction despite being utterly and unequivocally a bad choice for the very problem domain it’s intended to be used for. The historical accident of adoption and momentum in Javascript sets back professional computer science by decades until it’s eventually wholesale replaced with something whose first principles are actually appropriate.
That same reckoning is in flux in many fields of statistics, as the fundamental unreliability of NHST estimation is more understood and drop-in Bayesian replacements are more available.
Re: Time Series Prediction Using LSTM Deep Neural Networks
#57Earlier quoted context omitted.
I'm curious what resources you found useful to learn stats modelling and what sorts of approaches have been useful. On one hand, it's almost a tautoloy that specific models should be better than general models, but I worked on some 2d time series classification with a statistician and afterwards, for kicks, I replaced the entire thing with a CNN+LSTM and it worked just as well as the whole complicated model he had co…
I highly recommend this econometrics text for getting started with statistics: https://www.amazon.com/Principles-Econometrics-5th-Carter-Hi... For modeling I found Wooldridge's panel and cross-section data book very useful: https://www.amazon.com/Econometric-Analysis-Cross-Section-Pa... Greene is a really useful reference text: https://www.amazon.com/Econometric-Analysis-8th-William-Gree... For advanced stats theory,…
I add any of Ken Train's work to this mix, especially on estimating discrete choice theory.
Re: Time Series Prediction Using LSTM Deep Neural Networks
#58I find it interesting that Computer Scientists are basically rediscovering statistics. Now when predicting time series, an issue is that most model (like ARIMA, GARCH etc.) are short-memory processes. When you look at the full-series prediction of LSTMs, you observe the same thing. So in terms of Time Series, Machine Learning is currently in the mid to late 80's compared to Financial Econometrics. So if you are a CS,…
Re: Time Series Prediction Using LSTM Deep Neural Networks
#59Earlier quoted context omitted.
No, it's the same. It is just about having access to control variables.
What Chomsky is saying is that the control variables don't exist until you create them because the most telling things don't happen until you have a specific hypothesis and make them happen to test the hypothesis.
Re: Time Series Prediction Using LSTM Deep Neural Networks
#60Earlier quoted context omitted.
I really violently oppose this characterization of ML as "just" curve fitting, as if curve fitting is some simple solved problem. It seems like there is a ignorance about issues relating to model selection, which is an essential part of curve fitting. What complexity of model does the data support? Can you keep a distribution over structures that allows uncertain parts of the model to be interrogated? These are the p…
But what you describe is still curve fitting. I say this in spite of some expertise in ML myself. There are some parts of ML that are not fall in the curve fitting family but they are still a small part, for example Markov logic network, some parts of reinforcement learning. What you are saying is curve fitting with good predictive ability is not trivial, and that is indeed true.