21–30 of 90 posts
Has anyone else struggled with Prophet? I've experimented with it on a few real world datasets and I've had very inconsistent results.
Earlier quoted context omitted.
Yes. I've tried using it for pretty straightforward time series forecasts, and I struggled to make it into something useful in a business context. I'll disclaim that I'm just a finance dude and not a data scientist or programmer. But the documentation leads me to believe that I am in the target audience. I felt like I could grasp the basic mechanics after reading the paper, but I wish the documentation could help som…
You are the primary audience. Time series forecasting with deep learning is fraught with inconsistency. Someone on r/ML went pretty hard on detailing a survey and the stuff that was SOTA 10 years ago still is. Wish I saved that thread. The dude was well published. edit: found it https://www.reddit.com/r/MachineLearning/comments/pe1lst/r_i... Turns out it was about time series anomaly detection, but if you can detect,…
Earlier quoted context omitted.
Isn't recommended by whom?
Every time I, or someone at work with more experience than me, have tried Prophet it has ended up in changing the approach and trying a different technique. In my experience with time series hand-crafted recipes tend to work much more better than out-of-the-box solutions.
Also relevant: https://news.ycombinator.com/item?id=27695574
“You can imagine my disappointment when, out-of-the-box, Prophet was beaten soundly by a ‘take the last value’ forecast.”
They recommend checking out these for cutting-edge time series forecasting:
Earlier quoted context omitted.
Yes. I've tried using it for pretty straightforward time series forecasts, and I struggled to make it into something useful in a business context. I'll disclaim that I'm just a finance dude and not a data scientist or programmer. But the documentation leads me to believe that I am in the target audience. I felt like I could grasp the basic mechanics after reading the paper, but I wish the documentation could help som…
You are the primary audience. Time series forecasting with deep learning is fraught with inconsistency. Someone on r/ML went pretty hard on detailing a survey and the stuff that was SOTA 10 years ago still is. Wish I saved that thread. The dude was well published. edit: found it https://www.reddit.com/r/MachineLearning/comments/pe1lst/r_i... Turns out it was about time series anomaly detection, but if you can detect,…
If I could build it again, I’d start with automating the evaluation of forecasts. It’s silly to build models if you’re not willing to commit to an evaluation procedure. I’d also probably remove most of the automation of the modeling. People should explicitly make these choices.
Even in the age of deep learning, I do think these relatively classical Bayesian models have a lot of value for many applications. Time-series forecasting tends to be a case where:
- you don't have a ton of iid data points (often, only a single time series),
- you'd like forecasts with principled uncertainty estimates, e.g., credible intervals, giving you a range of scenarios to plan for,
- you often do have a pretty good idea of what features are relevant to the process you're predicting, and
- you want to understand in detail what features the forecast is accounting for (and what it might be missing),
all of which play to the strengths of more classical, structured statistical models, compared to more data-hungry black-box deep learning models. So the basic ideas in Prophet and similar tools do still have a lot of relevance going forward, IMHO.
The problem with time series forecasting in general is that they make a lot of assumptions on the shape of your data, and you'll find you're spending a lot of time figuring out mutating your data. For example, they expect that your data comes at a very regular interval. This is fine if it's, say, the data from a weather station. This doesn't work well in clinical settings (imagine a patient admitted into the ER -- there is a burst of data, followed by no data).
That said, there's some interesting stuff out there that I've been experimenting with that seems to be more tolerant of irregular time series and can be quite useful. If you're interested in exchanging ideas, drop me a line (email in my profile).