weather prediction seems to me like a terrific use of machine learning aka statistics. The challenge I suppose is in the data. To get perfect predictions you'd need to have a mapping of what conditions were like 6 hours, 12 hours, etc before, and what the various outcomes were, which butterflies flapped their wings and where (this last one is a joke about how hard this data would be). Hard but not impossible. Maybe impossible. I know very little about weather data though. Is there already such a format?
GraphCast: AI model for weather forecasting
11–20 of 310 posts
Re: GraphCast: AI model for weather forecasting
#12This is fascinating: > For inputs, GraphCast requires just two sets of data: the state of the weather 6 hours ago, and the current state of the weather. The model then predicts the weather 6 hours in the future. This process can then be rolled forward in 6-hour increments to provide state-of-the-art forecasts up to 10 days in advance.
Weather is markovian
Re: GraphCast: AI model for weather forecasting
#13Earlier quoted context omitted.
DeepMind recently merged with the Brain team from Google Research to form `Google DeepMind`. It seems this was done to have Google DeepMind focused primarily (only?) on AI research, leaving Google Research to work on other things in more than 20 research areas. Still, some AI research involves both orgs, including MetNet in weather forecasting. In any case, GraphCast is a 10-day global model, whereas MetNet is a 24-h…
Good explanation. Now that both the 24-hour regional and 10-day global models have been announced in technical/research detail, I supposed there might still be a general blog post about how improved forecasting is when you search for "weather" or check the forecast on Android.
That would require your local weather service to use these models
Re: GraphCast: AI model for weather forecasting
#14This is fascinating: > For inputs, GraphCast requires just two sets of data: the state of the weather 6 hours ago, and the current state of the weather. The model then predicts the weather 6 hours in the future. This process can then be rolled forward in 6-hour increments to provide state-of-the-art forecasts up to 10 days in advance.
Weather is markovian
[deleted]
Re: GraphCast: AI model for weather forecasting
#15I live in an area which regularly has a climate differently then forecasted: often less rain and more sunny. Would be great if I can connect my local weather station (and/or its history) to some model and have more accurate forecasts.
One piece of context to note here is that models like ECMWF are used by forecasters as a tool to make predictions - they aren't taken as gospel, just another input.
The global models tend to consistently miss in places that have local weather "quirks" - which is why local forecasters tend to do better than, say, accuweather, where it just posts what the models say.
Local forecasters might have learned over time that, in early Autumn, the models tend to overpredict rain, and so when they give their forecasts, they'll tweak the predictions based on the model tendencies.
Re: GraphCast: AI model for weather forecasting
#16To call this impressive is an understatement. Using a single GPU, outperforms models that run on the world's largest super computers. Completely open sourced - not just model weights. And fairly simple training / input data.
> ... with the current version being the largest we can practically fit under current engineering constraints, but which have potential to scale much further in the future with greater compute resources and higher resolution data.
I can't wait to see how far other people take this.
Re: GraphCast: AI model for weather forecasting
#17Earlier quoted context omitted.
DeepMind recently merged with the Brain team from Google Research to form `Google DeepMind`. It seems this was done to have Google DeepMind focused primarily (only?) on AI research, leaving Google Research to work on other things in more than 20 research areas. Still, some AI research involves both orgs, including MetNet in weather forecasting. In any case, GraphCast is a 10-day global model, whereas MetNet is a 24-h…
Is there a colab example (and/or have they released the models) for MetNet like they have here for GraphCast?
MetNet-3 is not open-source, and the announcement said it's already integrated into Google products/services needing weather info. So, I'd doubt there's anything like a colab example.
Re: GraphCast: AI model for weather forecasting
#18This is fascinating: > For inputs, GraphCast requires just two sets of data: the state of the weather 6 hours ago, and the current state of the weather. The model then predicts the weather 6 hours in the future. This process can then be rolled forward in 6-hour increments to provide state-of-the-art forecasts up to 10 days in advance.
Interesting indeed, only one lagged feature for time series forecasting? I’d imagine that including more lagged inputs would increase performance.
Rolling the forecasts forward to get n-step-ahead forecasts is a common approach. I’d be interested in how they mitigated the problem of the errors accumulating/compounding.
Re: GraphCast: AI model for weather forecasting
#19I've been following these global ML weather models. The fact they make good forecasts at all was very impressive. What is blowing my mind is how fast they run. It takes hours on giant super computers for numerical weather prediction models to forecast the entire globe. These ML models are taking minutes or seconds. This is potentially huge for operational forecasting. Weather forecasting has been moving focus towards…
It uses era5 data which is reanalysis. These models will always need the numerical training data. What's impressive is how well the emulate the physics in those models so cheaply. But since the climate changes there will eventually be different weather in different places.
https://www.ecmwf.int/en/forecasts/documentation-and-support
Re: GraphCast: AI model for weather forecasting
#20In case someone is looking for historical weather data for ML training and prediction, I created an open-source weather API which continuously archives weather data.
Using past and forecast data from multiple numerical weather models can be combined using ML to achieve better forecast skill than any individual model. Because each model is physically bound, the resulting ML model should be stable.