Live data from Hacker News

Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

microprediction.com

21–30 of 82 posts

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#21
post #18

I work with highly seasonal data (city-wide water consumption) and Prophet has been a great tool for us. In terms of performance, it has been the best for a few of our forecasts, compared to GRUs, LSTM, ARIMA and SARIMA. When it wasn't the best, it wasn't too far from the best model. But, to be fair, our forecast are of quite stable data, so most models do well. However, I would say that the key strength of Prophet i…

How well does it work with irregularly spaced time series?

Just as well, in my experience, since it's curve-fitting and not autoregressive.

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#22
If you are interested in time series predictions, I would suggest you had a look at darts (https://github.com/unit8co/darts). It's a well designed library which provides a unified API to deal with time series and try/compare different algorithms/frameworks like Prophet, recurrent NNs, etc.

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#23
post #17
post #14

My experience in general is that most time series model are inadequate in predicting time series except for very trivial cases of seasonalities or simple linear/nonlinear trends. I think that you can throw any model you like at the problem but all you will do is overfit most of the time.

Personally I find there is one important factor, and one factor alone. Context. Seasonality is the context that there's a seasonal driver at play. The context of a public holiday can explain a decrease in sales on that day. The context of a football match can explain a spike in transport demand near a stadium. The context of the presence of a heat dome predicted by pressure data can explain record temperature figures…

Is there also "one factor alone" in predicting stock market performance? Foreign-exchange rates? Lottery numbers?

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#24
i've used prophet along many other ts methods for price forecasting in energy trading. my experience is that prophet is ok, but rather opaque. having tried many packages, I've always come back to the classical statistical methods seeing benefits in transparency (what's actually going on? impact of regressors?), speed and most importantly that these methods force the user to think about what's happening in the data and make conscious decisions about how to model things. but i can see that sometimes you dont care too much about accuracy and understanding but that you just want a forecast for something that works decently without much hassle.

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#25
post #23
post #17

Earlier quoted context omitted.

Personally I find there is one important factor, and one factor alone. Context. Seasonality is the context that there's a seasonal driver at play. The context of a public holiday can explain a decrease in sales on that day. The context of a football match can explain a spike in transport demand near a stadium. The context of the presence of a heat dome predicted by pressure data can explain record temperature figures…

Is there also "one factor alone" in predicting stock market performance? Foreign-exchange rates? Lottery numbers?

Stock market is anti-inductive: future performance takes into account your attempts at predicting it. Any regularity in the stop market disappears the moment someone spots it and starts trading on it.

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#26

Earlier quoted context omitted.

How well does it work with irregularly spaced time series?

Just as well, in my experience, since it's curve-fitting and not autoregressive.

Does it apply any kind of persistence/memory on the instantaneous exogenous variables you feed it? E.g. if you feed it the exogenous variable of "temperature right now", is it able to create a new exogenous feature "average temperature over the last three time steps"?

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#28
I'm a professional forecaster (i.e. getting paid for it) at a large e-commerce company. We have extensive experience with Prophet and a host of other approaches (all the traditional models in Hyndman's book/R package, some scattered LSTM/NN implementations). Here's my quick take (the article is a lot more extensive than the median blogpost, and likely warrants a more extensive study than I have time for right now.)

Prophet main claims ("Get a reasonable forecast on messy data with no manual effort. Prophet is robust to outliers, missing data, and dramatic changes in your time series.") are surely exaggerated. As the article shows, time series come in many different shapes, and many of them are not handled properly. It deals well with distant-past or middle-of-the-sample outliers, but not with recent outliers. It cannot deal with level changes (as opposed to trend/slope changes). None of this should be a surprise if you take some time to understand the underlying model, which unlike most neural nets is very easily to completely understand and visualise: it's really a linear regression model with fixed-frequency periodic components (for yearly seasonality and weekly seasonality) and a somewhat-flexible piecewise-linear trend. The strong assumption that the trend is continuous (with flexible slopes that pivot around a grid of trend breakpoints, which are trimmed by regularisation) accounts for most of the cases where the forecasts are clearly wrong.

That said, it does occupy a bit of a sweet spot in commercial forecasting applications. It it's largely tuned for a few years of daily data with strong and regular weekly and yearly seasonalities (and known holidays), or a few weeks/months of intraday and weekday seasonalities. Such series are abundant in commerce, but a bit of a weak spot for the traditional ARIMA and seasonal exponential smoothers in Hyndman's R package. These tended to be tuned on monthly or quarterly data, where Prophet often performs worse. In our experience, for multiple years of daily commercial-activity data, there are no automated approaches that easily outperform Prophet. You can get pretty similar (or slightly better) results with Hyndman's TBATS model if you choose the periodicities properly (not surprising, as the underlying trend-season-weekday model is pretty similar as Prophet, but a bit more sophisticated). Some easy win for the Prophet devs are probably to incorporate a Box-Cox step in the model, and a sort-term ARMA error correction, then the model really resembles TBATS. You can usually get better results with NNs that are a bit more tuned to the dataset. But if you know nothing a priori about the data except that it's a few years of sales data, your fancy NN will probably resemble Prophet's trend-season-weekday model anyway.

All of these assume that we're trying to forecast any time series' future only from its own past. If you want to predict (multiple) time series using multiple series as input/predictors, that's a whole new level of difficulty. I don't know of a good automatic/fast/scalable approach that properly guards against overfitting. Good results for multiple-input forecasting approaches probably requires some amount of non-scalable "domain knowledge".

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#29

Earlier quoted context omitted.

Just as well, in my experience, since it's curve-fitting and not autoregressive.

Does it apply any kind of persistence/memory on the instantaneous exogenous variables you feed it? E.g. if you feed it the exogenous variable of "temperature right now", is it able to create a new exogenous feature "average temperature over the last three time steps"?

No, you have to handcraft all (transformations) of exogenous features. But since it's really all linear regression, that's usually reasonably straightforward.
Post reply on HN