As someone who spent a good part of my professional career in forecasting and time-series analysis I would like to point out that "point-forecasts" are mostly useless in many import practical applications such as FinTech, e-commerce, sports betting, etc. Point-forecast models such as Prophet fail to give you a meaningful measure of uncertainty of the predicted value. A much better approach are probabilistic forecasti…
Can you elaborate a bit? Prophet can use MCMC sampling and includes uncertainty in its forecasts.
Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?
51–60 of 82 posts
Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?
#52As someone who spent a good part of my professional career in forecasting and time-series analysis I would like to point out that "point-forecasts" are mostly useless in many import practical applications such as FinTech, e-commerce, sports betting, etc. Point-forecast models such as Prophet fail to give you a meaningful measure of uncertainty of the predicted value. A much better approach are probabilistic forecasti…
Do you have any favourite libraries for producing such?
For modern Deep Learning based probabilistic forecasting you can try DeepAR with parametric likelihood function [0] or Multi-Horizon Quantile RNN (non-parametric) [1]. The implementations of these models in Pytorch and MXnet are scattered all over the place.
[0] https://arxiv.org/abs/1704.04110
[1] https://arxiv.org/abs/1711.11053
EDIT: formatting
Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?
#53(from the article)
- The author
Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?
#54My experience in general is that most time series model are inadequate in predicting time series except for very trivial cases of seasonalities or simple linear/nonlinear trends. I think that you can throw any model you like at the problem but all you will do is overfit most of the time.
In my experience, the time-series models that are reliably predictive typically aren't purely autoregressive but contain exogeneous variables as well (i.e. yₖ = f(yₖ₋₁, yₖ₋₂, ..., xₖ, xₖ₋₁, xₖ₋₂...), like ARX models). These models don't only capture relationships to historical patterns but to other driving/causal variables.
Price forecasts are often modeled as time-series models, but this assumes that prices only have time-based patterns which is often not true. In my domains of interest for instance, time has tangible yet limited effect on prices -- prices are driven more by variables like weather and certain types of market activity.
Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?
#55Earlier quoted context omitted.
Can you elaborate a bit? Prophet can use MCMC sampling and includes uncertainty in its forecasts.
Prophet with MCMC can produce probabilistic forecasts. But how to choose MCMC priors and measure accuracy of the probabilistic forecast are open questions.
Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?
#56Prophet solves a broad class of easy problems that a lot of ordinary businesses have: you have several years of basically regular data (sales or page views or store foot traffic) that you know has yearly/weekly/daily (if you have sub-daily data) cycles, and you want to give a reasonable prediction to the business so they can plan for the upcoming week/month/year. And you want to remove the periodic effects so you can see the underlying trends.
Imagine someone, lets call them Bill, who might be called a data scientist, or business analyst or just assistant operations manager, for a medium-large business. Bill has the last 5 years of sales/views/traffic data in the database (anything before that is in a bunch of excel spreadsheets on the share drive), and knows just enough python to be dangerous. Bill can probably explain an R-squared value but is not an expert at statistics by any measure. He wants to fit the data, but has several problems:
1) the weekly trend does not line up with the yearly data, as the year starts on a different weekday.
2) Those damn public holidays, some of them occur on a specific date, some of them on the "first Monday of the month", and some of them seem to change almost randomly year-to-year.
3) The reporting system was down for a couple of weeks in June and Feb last year, and the numbers for the first few years were copied from excel, so sometimes are missing the first or last day of the month.
Prophet comes by default with yearly/weekly seasonality. Prophet comes out-of-the-box with a simple way to import holidays, and even a way to specify your own. Prophet doesn't require any cleaning, or special procedures to deal with missing data. And it is quick and easy to use, and get nice-looking, broadly reasonable graphs out (with the above mentioned, consistent data). And that solves the business problem.
And Bill's probably heard of it because it is (a) popular already, and (b) has Facebook's name attached.
That's my take as to why, even if it is not even close to the most accurate method, Prophet is so broadly popular.
Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?
#57My experience in general is that most time series model are inadequate in predicting time series except for very trivial cases of seasonalities or simple linear/nonlinear trends. I think that you can throw any model you like at the problem but all you will do is overfit most of the time.
Personally I find there is one important factor, and one factor alone. Context. Seasonality is the context that there's a seasonal driver at play. The context of a public holiday can explain a decrease in sales on that day. The context of a football match can explain a spike in transport demand near a stadium. The context of the presence of a heat dome predicted by pressure data can explain record temperature figures…
As an aside, the way this is handled is with a calendar like the https://nrf.com/resources/4-5-4-calendar
I wrote a time library once that handled calendars like this automagically, so you can roll the context in if you're clever enough.
Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?
#58As someone who spent a good part of my professional career in forecasting and time-series analysis I would like to point out that "point-forecasts" are mostly useless in many import practical applications such as FinTech, e-commerce, sports betting, etc. Point-forecast models such as Prophet fail to give you a meaningful measure of uncertainty of the predicted value. A much better approach are probabilistic forecasti…
[0] https://twimlai.com/causal-models-in-practice-at-lyft-with-s...
Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?
#59Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?
#60To close, let me say that this post ended up being more negative than I expected and, like Nessie, my opinion may rise in the future when I understand the implications of the Prophet generative model better, and either modify it or find better ways to identify its strengths. The unanswered question here is why Prophet is so popular, and this surely merits a better explanation than I have given. I think there are prob…