Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GraphCast: AI model for weather forecasting

deepmind.google

Post reply on HN
Print view View original

91–100 of 310 posts

Re: GraphCast: AI model for weather forecasting

#91

Post by dekhn »

serjester wrote: post #41
silveraxe93 wrote: post #38

Earlier quoted context omitted.

Could you point me to the part where it says it depends on supercomputer output? I didn't read the paper but the linked post seems to say otherwise? It mentions it used the supercomputer output to impute data during training. But for prediction it just needs: > For inputs, GraphCast requires just two sets of data: the state of the weather 6 hours ago, and the current state of the weather. The model then predicts the…

You can read about it more in their paper. Specifically page 36. Their dataset, ERA5, is created using a process called reanalysis. It combines historical weather observations with modern weather models to create a consistent record of past weather conditions. https://storage.googleapis.com/deepmind-media/DeepMind.com/B...

I can't find the details, but if the supercomputer job only had to run once, or a few times, while this model can make accurate predictions repeatedly on unique situations, then it doesn't matter as much that a supercomputer was required. The goal is to use the supercomputer once, to create a high value simulated dataset, then repeatedly make predictions from the lower-cost models.

Re: GraphCast: AI model for weather forecasting

#92

Post by joaogui1 »

pyb wrote: post #88
keule wrote: post #86

Earlier quoted context omitted.

IMO a chaotic system will not allow for long-term forecast, but if there is any type of pattern to recognize (and I would assume there are plenty), an AI/ML model should be able to create short-term prediction with high accuracy.

Not an expert, but "Up to 10 days in advance" sounds like long-term to me ?

I think 10 days is basically the normal term for weather, in that we can get decent predictions for that span using "classical"/non-ML methods.

Re: GraphCast: AI model for weather forecasting

#93

Post by dmd »

meteo-jeff wrote: post #42
mdbmdb wrote: post #34

Earlier quoted context omitted.

Is it able to provide data on extreme events. Say, the current and potential path of a hurricane? similar to .kml that NOAA provides

Extreme weather is predicted by numerical weather models. Correctly representing hurricanes has driven development on the NOAA GFS model for centuries. Open-Meteo focuses on providing access to weather data for single locations or small areas. If you look at data for coastal areas, forecast and past weather data will show severe winds. Storm tracks or maps are not available, but might be implemented in the future.

I would love to hear about this centuries-old NOAA GFS model. The one I know about definitely doesn't have that kind of history behind it.

Re: GraphCast: AI model for weather forecasting

#94

Post by joaogui1 »

kouru225 wrote: post #89
keule wrote: post #86

Earlier quoted context omitted.

IMO a chaotic system will not allow for long-term forecast, but if there is any type of pattern to recognize (and I would assume there are plenty), an AI/ML model should be able to create short-term prediction with high accuracy.

But AI/ML models require good data and the issue with chaotic systems like weather is that we don’t have good enough data.

The issue with chaotic systems is not data, is that the error grows superlinearly with time, and since you always start with some kind of error (normally due to measurement limitations) this means that after a certain time horizon the error becomes to significant to trust the prediction. That hasn't a lot to do with data quality for ML models

Re: GraphCast: AI model for weather forecasting

#95

Post by kouru225 »

joaogui1 wrote: post #94
kouru225 wrote: post #89

Earlier quoted context omitted.

But AI/ML models require good data and the issue with chaotic systems like weather is that we don’t have good enough data.

The issue with chaotic systems is not data, is that the error grows superlinearly with time, and since you always start with some kind of error (normally due to measurement limitations) this means that after a certain time horizon the error becomes to significant to trust the prediction. That hasn't a lot to do with data quality for ML models

That’s an issue with data: If your initial conditions are wrong (Aka your data collection has any error or isn’t thorough enough) then you get a completely different result.

Re: GraphCast: AI model for weather forecasting

#96

Post by meteo-jeff »

meteo-jeff wrote: post #42
mdbmdb wrote: post #34

Earlier quoted context omitted.

Is it able to provide data on extreme events. Say, the current and potential path of a hurricane? similar to .kml that NOAA provides

Extreme weather is predicted by numerical weather models. Correctly representing hurricanes has driven development on the NOAA GFS model for centuries. Open-Meteo focuses on providing access to weather data for single locations or small areas. If you look at data for coastal areas, forecast and past weather data will show severe winds. Storm tracks or maps are not available, but might be implemented in the future.

Sorry, decades.

KML files for storm tracks are still the best way to go. You could calculate storm tracks yourself for other weather models like DWD ICON, ECMWF IFS or MeteoFrance ARPEGE, but storm tracks based on GFS ensembles are easy to use with sufficient accuracy

Re: GraphCast: AI model for weather forecasting

#98

Post by pyb »

joaogui1 wrote: post #92
pyb wrote: post #88

Earlier quoted context omitted.

Not an expert, but "Up to 10 days in advance" sounds like long-term to me ?

I think 10 days is basically the normal term for weather, in that we can get decent predictions for that span using "classical"/non-ML methods.

IDK, I wouldn't plan a hike in the mountains based on 10-day predictions.

Re: GraphCast: AI model for weather forecasting

#99

Post by K2h »

dmd wrote: post #93
meteo-jeff wrote: post #42

Earlier quoted context omitted.

Extreme weather is predicted by numerical weather models. Correctly representing hurricanes has driven development on the NOAA GFS model for centuries. Open-Meteo focuses on providing access to weather data for single locations or small areas. If you look at data for coastal areas, forecast and past weather data will show severe winds. Storm tracks or maps are not available, but might be implemented in the future.

I would love to hear about this centuries-old NOAA GFS model. The one I know about definitely doesn't have that kind of history behind it.

Some of the oldest data may come from ships logs back to 1836

https://www.reuters.com/graphics/CLIMATE-CHANGE-ICE-SHIPLOGS...

Re: GraphCast: AI model for weather forecasting

#100

Post by pyb »

kouru225 wrote: post #87
pyb wrote: post #83

Curious. How can AI/ML perform on a problem that is, as far as I understand, inherently chaotic / unpredictable ? It sounds like a fundamental contradiction to me.

Yes. Very accurate as long as you don’t need to predict the unpredictable. So it’s useless. Edit: I do see a benefit to the idea if you compare it to the Chaos Theorists “gaining intuition” about systems.

IDK if it's useless, but it's counter-intuitive to me.
Post reply on HN