Live data from Hacker News

Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

microprediction.com

71–80 of 82 posts

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#71
Shouldn't time-based (weekly and holiday) and trend effects be multiplied rather than added?

For instance, if users spend twice as many hours on the weekend on a website, and the total number of users has doubled, then these effects multiply to give 4x visits than the baseline non-weekend.

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#72
post #42

As someone who spent a good part of my professional career in forecasting and time-series analysis I would like to point out that "point-forecasts" are mostly useless in many import practical applications such as FinTech, e-commerce, sports betting, etc. Point-forecast models such as Prophet fail to give you a meaningful measure of uncertainty of the predicted value. A much better approach are probabilistic forecasti…

This is a Bayesian model so it of course provides a probability distribution for both the parameters and the forecasted values.

P(model | historic_data) = P(historic_data | model) P(model) / (sum(model) P(historic_data | model) P(model))

P(future_data | historic_data) = sum(model) P(future_data | model) P(model | historic_data) = (sum(model) P(future_data | model) P(historic_data | model) P(model)) / (sum(model) P(historic_data | model) P(model))

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#73

Messiah? No. But one big plus that this article doesn't talk much about is how easy it is to get started with it even if you're a beginner. A few lines of code gets you a fitted model and some insightful plots. From these you can see if 1) it's doing great and you don't need to spend hours training some crazy transformer model, 2) It's got some flaws and you should maybe try something else (which you can now compare…

It's easy to get started yes and your point is taken WRT deep NN. In fairness, the first sentence in the article reads "Facebook's Prophet package aims to provide a simple, automated approach to prediction of a large number of different time series." However it's actually quicker to get started, one might argue, with other packages and its certainly quick to get an idea of which might be accurate: https://microprediction.github.io/timeseries-elo-ratings/htm...

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#74
post #34
post #28

I'm a professional forecaster (i.e. getting paid for it) at a large e-commerce company. We have extensive experience with Prophet and a host of other approaches (all the traditional models in Hyndman's book/R package, some scattered LSTM/NN implementations). Here's my quick take (the article is a lot more extensive than the median blogpost, and likely warrants a more extensive study than I have time for right now.) P…

> If you want to predict (multiple) time series using multiple series as input/predictors, that's a whole new level of difficulty. I don't know of a good automatic/fast/scalable approach that properly guards against overfitting Have you had a look at algorithms contained in pytorch forcasting? https://pytorch-forecasting.readthedocs.io/en/latest/

I'll try to add notebook examples at https://www.microprediction.com/blog/popular-timeseries-pack... and get some pytorch-forecasting Elo ratings going. As an aside, anyone who wants to see a particular approach get rated is welcome to help! It amounts to creating a "skater" which is a simple functional form of forecasting. https://github.com/microprediction/timemachines/tree/main/ti...

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#75
post #42

As someone who spent a good part of my professional career in forecasting and time-series analysis I would like to point out that "point-forecasts" are mostly useless in many import practical applications such as FinTech, e-commerce, sports betting, etc. Point-forecast models such as Prophet fail to give you a meaningful measure of uncertainty of the predicted value. A much better approach are probabilistic forecasti…

Fun fact: if you publish a data point regularly at https://www.microprediction.com/get-predictions then over time, distributional predictions will come to you. It's like a nano-market for live distributional prediction.

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#76
post #18

I work with highly seasonal data (city-wide water consumption) and Prophet has been a great tool for us. In terms of performance, it has been the best for a few of our forecasts, compared to GRUs, LSTM, ARIMA and SARIMA. When it wasn't the best, it wasn't too far from the best model. But, to be fair, our forecast are of quite stable data, so most models do well. However, I would say that the key strength of Prophet i…

Do you have any city data that could be published on an hourly basis? We'd soon see how prophet does against other approaches. Just like this example: https://www.microprediction.org/stream_dashboard.html?stream...

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#77

I'll try to give my perspective, though it's mostly expanding on em500's [1] 3rd paragraph. I think you're asking the wrong question about prophet - most users don't care "how good is it?" but instead "is it good enough?", and then "how easy is it to use?" Prophet solves a broad class of easy problems that a lot of ordinary businesses have: you have several years of basically regular data (sales or page views or stor…

I agree and I hear these anecdotal success stories. What I'd be more than happy to do is add some time series which are considered to be in Prophet's strong suit, to see where they end up on leaderboards like https://www.microprediction.org/stream_dashboard.html?stream... However the skeptic in me thinks that prophet might be performing a bit of a trick (not in any evil way) on the user ... convincing them visually that something is being done when really, the generative model means you have to get really lucky for it to add value. Happy to be convinced otherwise!

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#78
post #61
post #49

Earlier quoted context omitted.

Can you elaborate a bit? Prophet can use MCMC sampling and includes uncertainty in its forecasts.

> Can you elaborate a bit? Prophet can use MCMC sampling and includes uncertainty in its forecasts. Prophet is a GAM (Generalised Additive Model). It decomposes time series in additive components: trend, seasonality, holidays and noise. Most interesting time-series are not so simply decomposable. Making Prophet Bayesian and producing probabilistic forecast by MCMC sampling from trend/seasonality/holiday posteriors st…

Yes, that’s my understanding too. GAMs have structural limitations that harm flexibility and help interpretability.

One benefit of Bayesian models that they work with relatively little data - and generally provide greater uncertainty in those cases. Do you happen to know of some DL frameworks that behave similarly? I’m eager to learn.

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#79
post #70

To close, let me say that this post ended up being more negative than I expected and, like Nessie, my opinion may rise in the future when I understand the implications of the Prophet generative model better, and either modify it or find better ways to identify its strengths. The unanswered question here is why Prophet is so popular, and this surely merits a better explanation than I have given. I think there are prob…

> The unanswered question here is why Prophet is so popular, and this surely merits a better explanation than I have given. It has been explained by several posters. The goal of Prophet is to make time series accessible to non-experts, and the alternatives to Prophet are significantly more complicated. I say this as someone who has done pretty extensive work with time series for BigCorp's chain of retail stores.

A moving average is not complicated. A moving average is more accurate. So I don't believe it has been explained in this thread at all.

Re: Is Facebook's “Prophet” the time-series Messiah or just a naughty boy?

#80

To close, let me say that this post ended up being more negative than I expected and, like Nessie, my opinion may rise in the future when I understand the implications of the Prophet generative model better, and either modify it or find better ways to identify its strengths. The unanswered question here is why Prophet is so popular, and this surely merits a better explanation than I have given. I think there are prob…

The title is a bit strong too, but maybe that’s what you were going for.

Stronger than "Prophet"?

I began writing this post because I was working on integrating Prophet into a Python package I call time machines, which is my attempt to remove some ceremony from the use of forecasting packages and compare them. These power some bots that the prediction network (explained at www.microprediction.com if you are interested). How could I not include the most popular time series package?

I hope you interpret this post as nothing more than an attempt to understand the quizzical performance results, without denying the possible utility of Prophet or its strengths (if nothing else it might be classified as a change-point detection package). I mean seriously, can Prophet really be all that bad? At minimum, all those who downloaded Prophet are casting a vote for interpretability, scalability and good documentation - but perhaps accuracy as well in a manner that is hard to grasp quantitatively.

(from the article)

Post reply on HN