Is Facebook's “Prophet” the time-series Messiah or just a naughty boy? • Hacker News Board

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#61

Post by lr1970 » Thu, Jul 01, 2021, 3:13 PM UTC

glial wrote: post #49

lr1970 wrote: post #42

As someone who spent a good part of my professional career in forecasting and time-series analysis I would like to point out that "point-forecasts" are mostly useless in many import practical applications such as FinTech, e-commerce, sports betting, etc. Point-forecast models such as Prophet fail to give you a meaningful measure of uncertainty of the predicted value. A much better approach are probabilistic forecasti…

Can you elaborate a bit? Prophet can use MCMC sampling and includes uncertainty in its forecasts.

> Can you elaborate a bit? Prophet can use MCMC sampling and includes uncertainty in its forecasts.

Prophet is a GAM (Generalised Additive Model). It decomposes time series in additive components: trend, seasonality, holidays and noise. Most interesting time-series are not so simply decomposable. Making Prophet Bayesian and producing probabilistic forecast by MCMC sampling from trend/seasonality/holiday posteriors still keeps its GAM structure. Might be for a simple exploratory analysis Prophet is a good go-to tool but all the research action is now in Deep Learning Forecasting Models.

Also, IMHO, Prophet deals with individual TS and teaching it to produce vector forecast for multiple TSes at the same time is tricky (or not even possible).

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#62

Post by PaulHoule » Thu, Jul 01, 2021, 3:23 PM UTC

You'd think people who are hyping AI were trying to bring about the next "AI Winter".

People hear "FAANG" and it suspends their critical judgement.

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#63

Post by kqr » Thu, Jul 01, 2021, 3:23 PM UTC

lr1970 wrote: post #42

As someone who spent a good part of my professional career in forecasting and time-series analysis I would like to point out that "point-forecasts" are mostly useless in many import practical applications such as FinTech, e-commerce, sports betting, etc. Point-forecast models such as Prophet fail to give you a meaningful measure of uncertainty of the predicted value. A much better approach are probabilistic forecasti…

This is a good point and true for many things other than timeseries analysis. Any time uncertainty is involved the language of probability is the only thing that makes sense.

And as you say, what matters is generally not the specific values you are uncertain of, but what consequences they can potentially have for your decisions. To know this, you really have to know most values that are possible and how likely they are.

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#64

Post by [deleted] » Thu, Jul 01, 2021, 3:28 PM UTC

em500 wrote: post #29

fighterpilot wrote: post #26

Earlier quoted context omitted.

Does it apply any kind of persistence/memory on the instantaneous exogenous variables you feed it? E.g. if you feed it the exogenous variable of "temperature right now", is it able to create a new exogenous feature "average temperature over the last three time steps"?

No, you have to handcraft all (transformations) of exogenous features. But since it's really all linear regression, that's usually reasonably straightforward.

[deleted]

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#65

Post by frakt0x90 » Thu, Jul 01, 2021, 3:34 PM UTC

lr1970 wrote: post #52

dcl wrote: post #43

Earlier quoted context omitted.

Do you have any favourite libraries for producing such?

> Do you have any favourite libraries for producing such? For modern Deep Learning based probabilistic forecasting you can try DeepAR with parametric likelihood function [0] or Multi-Horizon Quantile RNN (non-parametric) [1]. The implementations of these models in Pytorch and MXnet are scattered all over the place. [0] https://arxiv.org/abs/1704.04110 [1] https://arxiv.org/abs/1711.11053 EDIT: formatting

And if you want to play with this the gluonts python library makes it very easy. It also includes some other time series models as well! I've been playing with it at work and it's enjoyable.

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#66

Post by CuriouslyC » Thu, Jul 01, 2021, 3:39 PM UTC

lr1970 wrote: post #42

As someone who spent a good part of my professional career in forecasting and time-series analysis I would like to point out that "point-forecasts" are mostly useless in many import practical applications such as FinTech, e-commerce, sports betting, etc. Point-forecast models such as Prophet fail to give you a meaningful measure of uncertainty of the predicted value. A much better approach are probabilistic forecasti…

Gaussian processes are great for this. Another easy option is to bootstrap the estimator, though for large models that isn't feasible.

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#67

Post by jdewsnip » Thu, Jul 01, 2021, 3:46 PM UTC

Like most models its data dependent. Had quite a lot of success (was paid) using it on data with multi-seasonality (daily plus seasonal trend) with regressors and change points, where there is not a lot of other options.

As its a General Additive Model you can decompose the prediction into parts put them in front of a non-technical user for validation i.e. show effect of daily seasonality, yearly, holidays and regressors. You could even use it to show visually where the model is going wrong for predictions on a blog post ;)

Is it the most accurate model on all time series? No but it is useful and good enough for certain use cases.

I find it quite interesting what you can do with about 100 lines of stan code. Here is good link on some one building prophet in pymc3 rather than stan to explain its innards.

https://www.ritchievink.com/blog/2018/10/09/build-facebooks-...

If you want something more flexible you can drop down to this level of code i.e. pymc3, pyro, tfp and bsts. If just want a univarate forecast then ensembles of state space methods are hard to beat as evidenced by the M competitions.

But It’s Tough to Make Predictions, Especially About the Future

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#68

Post by chroem- » Thu, Jul 01, 2021, 3:51 PM UTC

zwaps wrote: post #38

chroem- wrote: post #9

Earlier quoted context omitted.

Yes, it does. I'm not sure what the author is trying to prove, but he's feeding a bunch of non-seasonal data into a highly seasonal modeling technique, and then hurling insults when it doesn't work because that's not what it was originally designed for. I'm sorry, but it just seems childish.

But this seems to be a bit contradictory. Both facebook and you claim that prophet is a package to easily do better than other simple approaches, thus providing at least a good baseline for applied data people. By contrast, the article suggests that prophet does badly in many common forecasting situations. So while prophet may do fine with piecewise linear seasonal data, it apparently is far from a good default. Ther…

The vast majority of time series data that you would encounter in a business setting is seasonal. Most of the datasets he is using are tiny and they haven't had enough history for the seasonality to develop.

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#69

Post by [deleted] » Thu, Jul 01, 2021, 4:11 PM UTC

zwaps wrote: post #38

chroem- wrote: post #9

Earlier quoted context omitted.

Yes, it does. I'm not sure what the author is trying to prove, but he's feeding a bunch of non-seasonal data into a highly seasonal modeling technique, and then hurling insults when it doesn't work because that's not what it was originally designed for. I'm sorry, but it just seems childish.

But this seems to be a bit contradictory. Both facebook and you claim that prophet is a package to easily do better than other simple approaches, thus providing at least a good baseline for applied data people. By contrast, the article suggests that prophet does badly in many common forecasting situations. So while prophet may do fine with piecewise linear seasonal data, it apparently is far from a good default. Ther…

[deleted]

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#70

Post by chroem- » Thu, Jul 01, 2021, 4:14 PM UTC

microprediction wrote: post #53

To close, let me say that this post ended up being more negative than I expected and, like Nessie, my opinion may rise in the future when I understand the implications of the Prophet generative model better, and either modify it or find better ways to identify its strengths. The unanswered question here is why Prophet is so popular, and this surely merits a better explanation than I have given. I think there are prob…

> The unanswered question here is why Prophet is so popular, and this surely merits a better explanation than I have given.

It has been explained by several posters. The goal of Prophet is to make time series accessible to non-experts, and the alternatives to Prophet are significantly more complicated. I say this as someone who has done pretty extensive work with time series for BigCorp's chain of retail stores.