Live data from Hacker News

Prophet: Automatic Forecasting Procedure

github.com

11–20 of 90 posts

Re: Prophet: Automatic Forecasting Procedure

#11

Has anyone else struggled with Prophet? I've experimented with it on a few real world datasets and I've had very inconsistent results.

Based just on the documentation, it seems there are some assumptions they expect the data to adhere to, and if they don't apply then it would not produce good results.

Re: Prophet: Automatic Forecasting Procedure

#12

I'm no time series expert, but from my experience and what I've heard, using Prophet for time series forecasting isn't recommended. It often leads to less-than-ideal results. Curiously, in Medium-like (ie low effort) publications it's still the recommended way to tackle a forecasting problem. The promise of a model that can solve any time series problem sounds great, but not all that glitters is gold, and as you get…

Isn't recommended by whom?

Every time I, or someone at work with more experience than me, have tried Prophet it has ended up in changing the approach and trying a different technique. In my experience with time series hand-crafted recipes tend to work much more better than out-of-the-box solutions.

Re: Prophet: Automatic Forecasting Procedure

#13

Can someone explain why the "no free lunch theorem" does not cause problems here? https://en.wikipedia.org/wiki/No_free_lunch_theorem

Two explanations

First: Prophet is not actually "one model", it's closer to a non-parametric approach than just a single model type. This adds a lot of flexibility on the class of problems it can handle. With that said, Prophet is "flexible" not "universal". A time series of entirely random integers selected from range(0,10) will be handled quite poorly, but fortunately nobody cares about modeling this case.

Second: the same reason that only a small handful of possible stats/ML models get used on virtually all problems. Most problems which people solve with stats/ML share a number of common features which makes it appropriate to use the same model on them (the model's "assumptions"). Applications which don't have these features get treated as edge-cases and ignored, or you write a paper introducing a new type of model to handle it. Consider any ARIMA-type time series model. These are used all the time for many different problem spaces, and are going to do reasonably well on "most" "common" stochastic processes you encounter in "nature", because its constructed to resemble many types of natural processes. It's possible (trivial, even) to conceive of a stochastic process which ARIMA can't really handle (any non-stationary process will work), but in practice most things that ARIMA utterly fails for are not very interesting to model or we have models that work better for that case.

Re: Prophet: Automatic Forecasting Procedure

#14

I'm no time series expert, but from my experience and what I've heard, using Prophet for time series forecasting isn't recommended. It often leads to less-than-ideal results. Curiously, in Medium-like (ie low effort) publications it's still the recommended way to tackle a forecasting problem. The promise of a model that can solve any time series problem sounds great, but not all that glitters is gold, and as you get…

I used Prophet and personally I do not have any problems, but I agree with the criticism that the tool it’s extremely focused in ergonomics that abstracts important aspects of the tool that can be used to built better models [1].

[1] - https://ryxcommar.com/2021/11/06/zillow-prophet-time-series-...

Re: Prophet: Automatic Forecasting Procedure

#15

I'm no time series expert, but from my experience and what I've heard, using Prophet for time series forecasting isn't recommended. It often leads to less-than-ideal results. Curiously, in Medium-like (ie low effort) publications it's still the recommended way to tackle a forecasting problem. The promise of a model that can solve any time series problem sounds great, but not all that glitters is gold, and as you get…

I've honestly had consistently better results with standard regression models. I really love the idea of it, and maybe I need to be tuning it better somehow, but overall I haven't had a great experience.

Re: Prophet: Automatic Forecasting Procedure

#17
post #7

Has anyone else struggled with Prophet? I've experimented with it on a few real world datasets and I've had very inconsistent results.

Yes. I've tried using it for pretty straightforward time series forecasts, and I struggled to make it into something useful in a business context. I'll disclaim that I'm just a finance dude and not a data scientist or programmer. But the documentation leads me to believe that I am in the target audience. I felt like I could grasp the basic mechanics after reading the paper, but I wish the documentation could help som…

You are the primary audience. Time series forecasting with deep learning is fraught with inconsistency. Someone on r/ML went pretty hard on detailing a survey and the stuff that was SOTA 10 years ago still is. Wish I saved that thread. The dude was well published.

edit: found it https://www.reddit.com/r/MachineLearning/comments/pe1lst/r_i...

Turns out it was about time series anomaly detection, but if you can detect, you can forecast if your model is generative

Re: Prophet: Automatic Forecasting Procedure

#18
This library is old news? Is there anything new that they've added that's noteworthy to take it for another spin?

[disclaimer I'm a maintainer of Hamilton] Otherwise FYI Prophet gels well with https://github.com/DAGWorks-Inc/hamilton for setting up your features and dataset for fitting & prediction[/disclaimer].

Re: Prophet: Automatic Forecasting Procedure

#19
I am intrigued on how this would perform on astronomical data.

If anyone is not aware there are many periodic phenomena in astronomy - e.g. variable stars which can have periods from minutes to hundreds of days.

The description of this library sounds like it's very tied to the human world - talking about yearly, weekly and daily seasonality.

[Weirdly though, we do sometimes see variability on 'human' timescales in astronomical data series. If maintenance is carried out weekly on a Monday that can add a signal into the data through missing datapoints.]

Re: Prophet: Automatic Forecasting Procedure

#20
post #7

Has anyone else struggled with Prophet? I've experimented with it on a few real world datasets and I've had very inconsistent results.

Yes. I've tried using it for pretty straightforward time series forecasts, and I struggled to make it into something useful in a business context. I'll disclaim that I'm just a finance dude and not a data scientist or programmer. But the documentation leads me to believe that I am in the target audience. I felt like I could grasp the basic mechanics after reading the paper, but I wish the documentation could help som…

This looks to me like something they’d be using for internal capacity planning. If so, they’d be asking it questions like, “how much capacity do we build out for the upcoming holiday rush?” I wouldn't be surprised if financial datasets are very noisy compared to service capacity metrics. I didn’t read the paper though, maybe this is addressed and maybe I’m wrong about the use case! But stuff like the below from the docs reads like capacity planning tool to me:

> As an example, let’s look at a time series of the log daily page views for the Wikipedia page for Peyton Manning. We scraped this data using the Wikipediatrend package in R. Peyton Manning provides a nice example because it illustrates some of Prophet’s features, like multiple seasonality, changing growth rates, and the ability to model special days (such as Manning’s playoff and superbowl appearances).

Post reply on HN