71–80 of 85 posts
Similar in vein:
https://emiruz.com/post/2022-11-16-defect-detection/
Extremely simple PCA based defect detection significantly beats orders of magnitude more complex segmentation network.
The conclusion, that a low-complexity statistical ensemble is almost as good as a (computationally) complex Deep Learning model, should not come as a surprise, given the data. The dataset[1] used here are 3003 time series from the M3 competition ran by the International Journal of Forecasting. Almost all of these are sampled at the yearly, quarterly or monthly frequency, each with typically 40 to 120 observations ("s…
I'm heavily involved in this area of research (getting deep learning competitive with computationally efficient statistical methods), and I'd like to note a couple things I've found: 1. Deep learning doesn't require thorough understanding of priors or statistical techniques. This opens the door to more programmers in the same way high level languages empower far more people than pure assembly. The tradeoffs are analo…
I don’t say this naively. At least in industries, the weight of imposter data scientists I think is getting to a level that may cause the profession to implode due to customer disillusionment within the next 10 years precisely because fit(X,y) is so accessible.
Earlier quoted context omitted.
I don’t think I have read anything more false on the internet. XD
I apologize. ^This comment was to harsh. Statistics can be summarizes as one thing n -> N. Does ‘little n’ represent ‘big N’. In other words, does the sample generalize to the population. Statistics means something like “description of the state”. It was born out of census samples where larger population samples had to be estimated. “n” could be a handful of fish in a “N” lake. “n” could also be the parameter estimat…
It was actually very cool because the model was a melt of exponential smoothing and dl.
I've done some work in this area and have indeed found that simpler statistical models often out perform ML/DL. This is true for single time series, where we are predicting P(x_t+1 | x_0..t) DL has advantages when you a) have additional context at each time step, or b) you have multiple related time series. For example, consider Amazon who predicts demands for all of their products. At each time step, they know about…
My understanding is that the biggest advantage between traditional machine learning and neural networks is that neural networks are useful when features either need to be generated or are poorly understood (such as a binary blob of an image or sound sample). So for data that is already neatly organized and labeled in a spreadsheet, DL loses its main advantage.
Earlier quoted context omitted.
I apologize. ^This comment was to harsh. Statistics can be summarizes as one thing n -> N. Does ‘little n’ represent ‘big N’. In other words, does the sample generalize to the population. Statistics means something like “description of the state”. It was born out of census samples where larger population samples had to be estimated. “n” could be a handful of fish in a “N” lake. “n” could also be the parameter estimat…
A point estimate of distribution parameters describing a population is frequentist. A point estimate of distribution parameters describing another distribution's parameter is Bayesian.
In statistics there are latin letters and greek letters. When you see a symbol denoted as a greek letter then that is a population parameter. When you see a latin letter that is a sample estimate. It could be Frequentist, Bayesian, Likelihoodist, Fiducial, Empirical Bayes, etc. Theoretical population greeks or sample calculated latins.
The conclusion, that a low-complexity statistical ensemble is almost as good as a (computationally) complex Deep Learning model, should not come as a surprise, given the data. The dataset[1] used here are 3003 time series from the M3 competition ran by the International Journal of Forecasting. Almost all of these are sampled at the yearly, quarterly or monthly frequency, each with typically 40 to 120 observations ("s…
If the task is to predict the next 12 values from a sample of 120 previous values, drawn from some computationally simple statistical process, it's much cheaper and easier to use old-fashioned, tried-and-true statistical methods.
If the task is to predict millions of pixel values that make up an original work of art, or the pixel values over time that make up a deep-fake video, or the next set of values encoding the next best possible play in a game of Go, or the set of values that encode the entire structure of a protein, and so on, then you have no choice: You must use a deep neural network. Simple methods cannot do any of that.
Earlier quoted context omitted.
Question: why do we care about the Bonferroni correction if the model being reviewed shows high performance on holdout/test samples? I mean, it's nice to know that the p-values of coefficients on models you are submitting for publication are appropriately reported under the conservative approach Bonferroni applies, but I would think making it a _default_ is an inappropriate forcing function when the performance on ho…
The test sample is just a small, arbitrary sample from a universe of similar data. You (probably) don't care about test-set performance per se but instead want to be able to claim that one model works better _in general_ than another. For that, you need to bust out the tools of statistical inference.
This isn't always true (of course, especially in aggregate series over time) and of course statistical measures are used to report model performance. But a Bonferroni correction struck me as a weird place to apply this specifically, but after the other comments from yesterday I saw where they were taking it.
I've done some work in this area and have indeed found that simpler statistical models often out perform ML/DL. This is true for single time series, where we are predicting P(x_t+1 | x_0..t) DL has advantages when you a) have additional context at each time step, or b) you have multiple related time series. For example, consider Amazon who predicts demands for all of their products. At each time step, they know about…