Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GraphCast: AI model for weather forecasting

deepmind.google

Post reply on HN
Print view View original

201–210 of 310 posts

Re: GraphCast: AI model for weather forecasting

#201

Post by ryanlitalien »

willsmith72 wrote: post #126
meteo-jeff wrote: post #20

In case someone is looking for historical weather data for ML training and prediction, I created an open-source weather API which continuously archives weather data. Using past and forecast data from multiple numerical weather models can be combined using ML to achieve better forecast skill than any individual model. Because each model is physically bound, the resulting ML model should be stable. See: https://open-me…

this is really cool, I've been looking for good snow-related weather APIs for my business. I tried looking on the site, but how does it work, being coordinates-based? I'm used to working with different weather stations, e.g. seeing different snowfall prediction at the bottom of a mountain, halfway up, and at the top, where the coordinates are quite similar.

You'll need a local weather expert to assist, as terrain, geography and other hyper-local factors create forecasting unpredictability. For example, Jay Peak in VT has its own weather, the road in has no snow, but it's a raging snowstorm on the mountain.

Re: GraphCast: AI model for weather forecasting

#202

Post by ArnoVW »

mg wrote: post #197

It's interesting, that Google keeps publishing AI research papers. Is there a business rationale behind it? OpenAI has become one of the fastest growing companies of all time. And much of it is based on Google's "Attention is all you need" and other papers. Since Microsoft added the Dall-E 3 image creator to Bing, Bing saw a huge inflow of new users. Dall-E is also a technology rooted in Google papers. I wonder how G…

It’s difficult to retain top talent if you do not allow them to publish.

Re: GraphCast: AI model for weather forecasting

#203

Post by flir »

uoaei wrote: post #198
isaacfrond wrote: post #191

I find this quite surprising actually. You'd think predicting the weather is mostly a matter of fast computation. The physical rules are well understood, so to get a better estimate use a finer mesh in your finite element computation and use a smaller time scale in estimating your differential equations. Neural networks are notoriously bad at exact approximation. I mean you can never beat a calculator when the issue…

> The physical rules are well understood Nope. They're constantly updating these models with really finnicky things like cloud nucleation rates that differ depending on which tree species's pollen is in the air. They've gotten a lot better (~2 day to ~7 day hi-res forecasts) but they're still wrong a lot of the time. The reason is the chaos as you say, however, chaos is deterministic, so, that a deterministic method…

It's a model of a model?

And it turns out to be better?

That's so counter-intuitive I'm kinda amazed anyone even bothered to research it, let alone that it worked.

Uh..... now do horse racing.

Re: GraphCast: AI model for weather forecasting

#204

Post by Auracle »

berniedurfee wrote: post #156
Fatnino wrote: post #116

Earlier quoted context omitted.

Is there somewhere to see historical forecasts? So not "the weather on 25 December 2022 was such and such" but rather "on 20 December 2022 the forecast for 25 December 2022 was such and such"

I’ve always wanted to see something like that. I always wonder if forecasts are a coin flip beyond a window of a few hours.

I just quit photographing weddings (and other stuff) this year. It's a job where the forecast really impacts you, so you tend to pay attention.

The amount of brides I've had to calm down when rain was forecast for their day is pretty high. In my experience, in my region, precipitation forecasts more than 3 days out are worthless except for when it's supposed to rain for several days straight. Temperature/wind is better but it can still swing one way or the other significantly.

For other types of shoots I'd tell people that ideally we'd postpone on the day of, and only to start worrying about it the day before the shoot.

I'm in Minnesota, so our weather is quite a bit more dynamic than many regions, for what it's worth.

Re: GraphCast: AI model for weather forecasting

#205

Post by jedberg »

mg wrote: post #197

It's interesting, that Google keeps publishing AI research papers. Is there a business rationale behind it? OpenAI has become one of the fastest growing companies of all time. And much of it is based on Google's "Attention is all you need" and other papers. Since Microsoft added the Dall-E 3 image creator to Bing, Bing saw a huge inflow of new users. Dall-E is also a technology rooted in Google papers. I wonder how G…

For every paper they publish, they have three others that they are keeping to themselves. Publishing papers is a recruiting technique.

Re: GraphCast: AI model for weather forecasting

#206

Post by uoaei »

flir wrote: post #203
uoaei wrote: post #198

Earlier quoted context omitted.

> The physical rules are well understood Nope. They're constantly updating these models with really finnicky things like cloud nucleation rates that differ depending on which tree species's pollen is in the air. They've gotten a lot better (~2 day to ~7 day hi-res forecasts) but they're still wrong a lot of the time. The reason is the chaos as you say, however, chaos is deterministic, so, that a deterministic method…

It's a model of a model? And it turns out to be better? That's so counter-intuitive I'm kinda amazed anyone even bothered to research it, let alone that it worked. Uh..... now do horse racing.

"All models are wrong, some models are useful." Some are more wrong and more useful simultaneously ;) This is actually the typical state of things in numerical simulation: we have infinite-resolution differential equations modeling such physical systems, but to implement them in silico we need to discretize and approximate various aspects of those models to achieve usefulness re: time and accuracy. Google has merely gone one level further in the tradeoff.

For more info on Google's approach, look into surrogate models. It's becoming more common especially in things like weather and geology.

Re: GraphCast: AI model for weather forecasting

#207

Post by CorrectHorseBat »

matsemann wrote: post #199

How's the distribution of the errors? For instance I don't care if it's better on average by 1 Celsius each day for normal weather, if it once every month is off by 10 Celsius when there is a drastic weather event, for instance. I'm all for better weather data, it's quite critical up in the mountains, so that's why my question about how reliable it is in life&death situations.

https://www.science.org/doi/10.1126/science.adi2336

Seems like it's better at predicting extreme weather events

Re: GraphCast: AI model for weather forecasting

#209

Post by scellus »

EricLeer wrote: post #176

I am in the power forecasting domain, where weather forecasts are one of the most important inputs. What I find surprising is that with all the papers and publications from google in the past years, there seems to be no way to get access to these forecasts! We've now evaluated numerous of the ai weather forecasting startups that are popping up everywhere and so far for all of them their claims fall flat on their face…

GraphCast, Pangu-Weather from Huawei, FourCastNet and EC's own AIFS are available on the ECMWF chart website https://charts.ecmwf.int, click "Machine learning models" on the left tab. (Clicking anything makes the URL very long.)

Some of these forecasts are also downloadable as data, but I don't know whether GraphCast is. Alternatively, if forecasts have a big economic value to you, loading latest ERA5 and the model code, and running it yourself should be relatively trivial? (I'm no expert on this, but I think that is ECMWF's aim, to distribute some of the models and initial states as easily runnable.)

Re: GraphCast: AI model for weather forecasting

#210

Post by DaiPlusPlus »

ArnoVW wrote: post #202
mg wrote: post #197

It's interesting, that Google keeps publishing AI research papers. Is there a business rationale behind it? OpenAI has become one of the fastest growing companies of all time. And much of it is based on Google's "Attention is all you need" and other papers. Since Microsoft added the Dall-E 3 image creator to Bing, Bing saw a huge inflow of new users. Dall-E is also a technology rooted in Google papers. I wonder how G…

It’s difficult to retain top talent if you do not allow them to publish.

How does Apple do it, if anyone knows? Apple is so loathe to keep their potential product plans hidden that AAPL employees aren’t even allowed to have GitHub accounts without mgr approval… but they have to be employing serious researchers, but they’ll never get to publish on volition.
Post reply on HN