Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GraphCast: AI model for weather forecasting

deepmind.google

Post reply on HN
Print view View original

141–150 of 310 posts

Re: GraphCast: AI model for weather forecasting

#141

Post by counters »

supdudesupdude wrote: post #121
haolez wrote: post #79

Are there any experts around that can chime in on the possible impacts of this technology if widely adopted?

It doesnt predict rainfall so i doubt most of us will actually care about it until then. Still it depends on input data (the current state of weather etc). How are we supposed to accurately model the weather at every point in the world? Especially when tech bro Joe living in San Fran expects things to be accurate to a meter within his doorstep

GraphCast does predict rainfall - see https://charts.ecmwf.int/products/graphcast_medium-rain-acc?... for example.

Re: GraphCast: AI model for weather forecasting

#142

Post by counters »

amluto wrote: post #134
WhitneyLand wrote: post #49

Earlier quoted context omitted.

How does it make sense to say this is something you’ve “never studied”, followed by how they “ought to be” doing it better? It also seems like some of your facts differ from theirs, may I ask how far you read into the paper?

I read a decent amount of the paper, although not the specific details of the model they used. And when I say I "never studied" it, I mean that I never took a class or read a textbook. I do, in fact, know something about physics and fluids, and I have even personally done some fluid simulation work. There are perfectly good models for weather in an abstract sense: Navier-Stokes plus various chemical models plus heat…

Weather models are routinely run at resolutions as fine as 1-3 km - fine enough that we do not parameterize things like convection and allow the model to resolve these motions on its native grid. We typically do this over limited areas (e.g. domain the size of a continent), but plenty of groups have such simulations globally. It's just not practical (cost for compute and resulting data) to do this regularly, and it offers little by way of direct improvement in forecast quality.

Furthermore, we don't have to necessarily measure the whole atmosphere in 3D; physical constraints arising from Navier-Stokes still apply, and we use them in conjunction with the data we _do_ have to estimate a full 3D atmospheric state complete with uncertainties.

Re: GraphCast: AI model for weather forecasting

#143

Post by crazygringo »

Making progress on weather forecasting is amazing, and it's been interesting to see the big tech companies get into this space.

Apple moved from using The Weather Channel to their own forecasting a year ago [1].

Using AI to produce better weather forecasts is exactly the kind of thing that is right up Google's alley -- I'm very happy to see this, and can't wait for this to get built into our weather apps.

[1] https://en.wikipedia.org/wiki/Weather_(Apple)

Re: GraphCast: AI model for weather forecasting

#144

Post by crazygringo »

pyb wrote: post #83

Curious. How can AI/ML perform on a problem that is, as far as I understand, inherently chaotic / unpredictable ? It sounds like a fundamental contradiction to me.

Because there are tons of parts of weather where chaos isn't the limiting factor currently.

There are a limited number of weather stations producing measurements, and a limited "cell size" for being able to calculate forecasts quickly enough, and geographical factors that aren't perfectly accounted for in models.

AI is able to help substantially with all of these -- from interpolation to computational complexity to geography effects.

Re: GraphCast: AI model for weather forecasting

#145

Post by just_testing »

meteo-jeff wrote: post #20

In case someone is looking for historical weather data for ML training and prediction, I created an open-source weather API which continuously archives weather data. Using past and forecast data from multiple numerical weather models can be combined using ML to achieve better forecast skill than any individual model. Because each model is physically bound, the resulting ML model should be stable. See: https://open-me…

I was going to ask about air quality, but just opened the site and you have air quality as well! Thanks!

Re: GraphCast: AI model for weather forecasting

#146

Post by Vagantem »

polygamous_bat wrote: post #136
Vagantem wrote: post #123

Earlier quoted context omitted.

That’s awesome! I’ve hooked something similar up to my service - https://dropory.com which predicts which day it will rain the least for any location Based on historical data!

Yikes, after completed three steps I was asked for my email. No to your bait and switch, thanks!

It can take up to 10 min to generate a report - I had a spinner before but people just left the page. So I implemented a way to send it to them instead. I’ve never used the emails for anything else than that. Try it with a 10 min disposable email address if you like. Thanks for your feedback!

Re: GraphCast: AI model for weather forecasting

#147

Post by blacksmith_tb »

crazygringo wrote: post #143

Making progress on weather forecasting is amazing, and it's been interesting to see the big tech companies get into this space. Apple moved from using The Weather Channel to their own forecasting a year ago [1]. Using AI to produce better weather forecasts is exactly the kind of thing that is right up Google's alley -- I'm very happy to see this, and can't wait for this to get built into our weather apps. [1] https:/…

Well, Apple acquired Dark Sky and then shut it down for Android users[1], and then eventually for iOS users as well (but rolled it into the built in weather app, I think).

1: https://www.theverge.com/2020/3/31/21201666/apple-acquires-w...

Re: GraphCast: AI model for weather forecasting

#148

Post by _visgean »

pkulak wrote: post #52
wenc wrote: post #25

Earlier quoted context omitted.

It builds on top of supercomputer model output and does better at the specific task of medium term forecasts. It is a kind of iterative refinement on the data that supercomputers produce — it doesn’t supplant supercomputers. In fact the paper calls out that it has a hard dependency on the output produced by supercomputers.

Why can't they just train on historical data?

ERA5 is based on historical data. See it for yourself https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysi..., https://www.ecmwf.int/en/forecasts/dataset/ecmwf-reanalysis-...

I don't using raw historical data would work for any data intensive model - afaik the data is patchy - there are spots where we don't have that many datapoints - e.g. middle of ocean... Also there are new satelites that are only available for the last x years and you want to be able to use these for the new models. So you need a re-analysis of what it would look like if you had that data 40 years ago...

Also its very convinient dataset because many other models trained on it: https://github.com/google-research/weatherbench2 so easy to do benchmarking..

Re: GraphCast: AI model for weather forecasting

#149

Post by _visgean »

crazygringo wrote: post #143

Making progress on weather forecasting is amazing, and it's been interesting to see the big tech companies get into this space. Apple moved from using The Weather Channel to their own forecasting a year ago [1]. Using AI to produce better weather forecasts is exactly the kind of thing that is right up Google's alley -- I'm very happy to see this, and can't wait for this to get built into our weather apps. [1] https:/…

> Apple moved from using The Weather Channel to their own forecasting a year ago [1].

AFAIK they don't have their own forecasting models, they use same data sources as everyone else: https://support.apple.com/en-us/HT211777

Re: GraphCast: AI model for weather forecasting

#150

Post by _visgean »

brap wrote: post #104

Beyond the difficulty of running calculations (or even accurately measuring the current state), is there a reason to believe weather is unpredictable? I would imagine we probably have a solid mathematical model of how weather behaves, so given enough resources to measure and calculate, could you, in theory, predict the daily weather going 10 years into the future? Or is there something inherently “random” there?

See https://en.wikipedia.org/wiki/Numerical_weather_prediction

> Present understanding is that this chaotic behavior limits accurate forecasts to about 14 days even with accurate input data and a flawless model. In addition, the partial differential equations used in the model need to be supplemented with parameterizations for solar radiation, moist processes (clouds and precipitation), heat exchange, soil, vegetation, surface water, and the effects of terrain.

I think there is a hope that DL models wont have this problem.

Post reply on HN