Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GraphCast: AI model for weather forecasting

deepmind.google

Post reply on HN
Print view View original

181–190 of 310 posts

Re: GraphCast: AI model for weather forecasting

#181

Post by wenyuanyu »

kridsdale3 wrote: post #57
xnx wrote: post #7

Earlier quoted context omitted.

Good explanation. Now that both the 24-hour regional and 10-day global models have been announced in technical/research detail, I supposed there might still be a general blog post about how improved forecasting is when you search for "weather" or check the forecast on Android.

IIRC the MetNet announcement a few weeks ago said that their model is now used when you literally Google your local weather. I don't think it's available yet to any API that third party weather apps pull from, so you'll have to keep searching "weather in Seattle" to see it.

Any idea why it is still showing the "weather.com" link next to the forecast?

Re: GraphCast: AI model for weather forecasting

#182

Post by tomaskafka »

meteo-jeff wrote: post #20

In case someone is looking for historical weather data for ML training and prediction, I created an open-source weather API which continuously archives weather data. Using past and forecast data from multiple numerical weather models can be combined using ML to achieve better forecast skill than any individual model. Because each model is physically bound, the resulting ML model should be stable. See: https://open-me…

I confirm, open-meteo is awesome and has a great API (and API playground!). And is the only source I know to offer 2 weeks of hourly forecasts (I understand at that point they are more likely to just show a general trend, but it still looks spectacular).

It's a pleasure being able to use it in https://weathergraph.app

Re: GraphCast: AI model for weather forecasting

#183

Post by HereBePandas »

whoislewys_1 wrote: post #160

Predicting weather and stock prices don't seem too far apart. Is it inevitable that all market alpha gets mined by AI?

I'd be shocked - given the incentives - if it hasn't already happened to a great extent. Many of the types of people Google DeepMind hires are also the types of people hedge funds hire.

Re: GraphCast: AI model for weather forecasting

#184

Post by wodenokoto »

brna wrote: post #175
meteo-jeff wrote: post #20

In case someone is looking for historical weather data for ML training and prediction, I created an open-source weather API which continuously archives weather data. Using past and forecast data from multiple numerical weather models can be combined using ML to achieve better forecast skill than any individual model. Because each model is physically bound, the resulting ML model should be stable. See: https://open-me…

Hi Jeff, Great work, Respect! I just hit the daily limit on the second request at https://climate-api.open-meteo.com/v1/climate I see the limit for non-commercial use should be "less than 10.000 daily API calls". Technically 2 is less than 10.000, I know, but still I decided to drop you a comment. :)

10.000 requests / (24 hours * 60 minutes * 60 seconds) = 0.11 requests / second

or 1 request every ~9 seconds.

Maybe you just didn't space them enough.

Re: GraphCast: AI model for weather forecasting

#185

Post by meteo-jeff »

jjp wrote: post #179
Fatnino wrote: post #116

Earlier quoted context omitted.

Is there somewhere to see historical forecasts? So not "the weather on 25 December 2022 was such and such" but rather "on 20 December 2022 the forecast for 25 December 2022 was such and such"

Are you thinking something like https://www.forecastadvisor.com/ ?

I would like to see an independent forecast comparison tool similar to Forecast Advisor, which evaluates numerical weather models. However, getting reliable ground truth data on a global scale can be a challenge.

Since Open-Meteo continuously downloads every weather model run, the resulting time series closely resembles assimilated gridded data. GraphCast relies on the same data to initialize each weather model run. By comparing past forecasts to future assimilated data, we can assess how much a weather model deviates from the "truth," eliminating the need for weather station data for comparison. This same principle is also applied to validate GraphCast.

Moreover, storing past weather model runs can enhance forecasts. For instance, if a weather model consistently predicts high temperatures for a specific large-scale weather pattern, a machine learning model (or a simple multilinear regression) can be trained to mitigate such biases. This improvement can be done for a single location with minimal computational effort.

Re: GraphCast: AI model for weather forecasting

#186

Post by brahbrah »

tomaskafka wrote: post #182
meteo-jeff wrote: post #20

In case someone is looking for historical weather data for ML training and prediction, I created an open-source weather API which continuously archives weather data. Using past and forecast data from multiple numerical weather models can be combined using ML to achieve better forecast skill than any individual model. Because each model is physically bound, the resulting ML model should be stable. See: https://open-me…

I confirm, open-meteo is awesome and has a great API (and API playground!). And is the only source I know to offer 2 weeks of hourly forecasts (I understand at that point they are more likely to just show a general trend, but it still looks spectacular). It's a pleasure being able to use it in https://weathergraph.app

> And is the only source I know to offer 2 weeks of hourly forecasts

Enjoy the data directly from the source producing them.

American weather agency: https://www.nco.ncep.noaa.gov/pmb/products/gfs/

European weather agency: https://www.ecmwf.int/en/forecasts/datasets/open-data

The data’s not necessarily east to work with, but it’s all there, and you get all the forecast ensembles (potential forecasted weather paths) too

Re: GraphCast: AI model for weather forecasting

#187

Post by brap »

counters wrote: post #108
brap wrote: post #104

Beyond the difficulty of running calculations (or even accurately measuring the current state), is there a reason to believe weather is unpredictable? I would imagine we probably have a solid mathematical model of how weather behaves, so given enough resources to measure and calculate, could you, in theory, predict the daily weather going 10 years into the future? Or is there something inherently “random” there?

What you're describing is effectively how climate models work; we run a physical model which solves the equations that govern how the atmosphere works out forward in time for very long time integrations. You get "daily weather" out as far as you choose to run the model. But this isn't a "weather forecast." Weather forecasting is an initial value problem - you care a great deal about how the weather will evolve from t…

So isn’t it just a problem of measurement then?

Say you had a massive array of billions of perfect sensors in different locations, and had all the computing power to process this data, would an N year daily forecast then be a solved problem?

For the sake of the argument I’m ignoring ”external” factors that could affect the weather (e.g meteors hitting earth, changes in man-made pollution, etc)

Re: GraphCast: AI model for weather forecasting

#188

Post by brna »

wodenokoto wrote: post #184
brna wrote: post #175

Earlier quoted context omitted.

Hi Jeff, Great work, Respect! I just hit the daily limit on the second request at https://climate-api.open-meteo.com/v1/climate I see the limit for non-commercial use should be "less than 10.000 daily API calls". Technically 2 is less than 10.000, I know, but still I decided to drop you a comment. :)

10.000 requests / (24 hours * 60 minutes * 60 seconds) = 0.11 requests / second or 1 request every ~9 seconds. Maybe you just didn't space them enough.

Maybe, that would be funny. ~7 requests per minute would be a more dev-friendly way of enforcing the same quota.

Re: GraphCast: AI model for weather forecasting

#189

Post by counters »

brap wrote: post #187
counters wrote: post #108

Earlier quoted context omitted.

What you're describing is effectively how climate models work; we run a physical model which solves the equations that govern how the atmosphere works out forward in time for very long time integrations. You get "daily weather" out as far as you choose to run the model. But this isn't a "weather forecast." Weather forecasting is an initial value problem - you care a great deal about how the weather will evolve from t…

So isn’t it just a problem of measurement then? Say you had a massive array of billions of perfect sensors in different locations, and had all the computing power to process this data, would an N year daily forecast then be a solved problem? For the sake of the argument I’m ignoring ”external” factors that could affect the weather (e.g meteors hitting earth, changes in man-made pollution, etc)

At that point you're slipping into Laplace's Demon.

In practical terms, we see predictability horizons get _shorter_ when we increase observation density and spatial resolution of our models, because more, small errors from slightly imperfect observations and models still cascade to larger scales.

Re: GraphCast: AI model for weather forecasting

#190

Post by rrr_oh_man »

mjs wrote: post #180
berniedurfee wrote: post #156

Earlier quoted context omitted.

I’ve always wanted to see something like that. I always wonder if forecasts are a coin flip beyond a window of a few hours.

Looks like https://sites.research.google/weatherbench/ attempts to "benchmark" different forecast models/systems. They're very cautious about naming a "best" model though! > Weather forecasting is a multi-faceted problem with a variety of use cases. No single metric fits all those use cases. Therefore,it is important to look at a number of different metrics and consider how the forecast will be applied.

That last paragraph sounds like something ChatGPT would write.
Post reply on HN