Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

WeatherNext 2: Our most advanced weather forecasting model

blog.google

Post reply on HN
Print view View original

131–140 of 140 posts

Re: WeatherNext 2: Our most advanced weather forecasting model

#131

Post by lysecret »

albertzeyer wrote: post #118
lysecret wrote: post #30

Im pretty deep into this topic and what might be interesting to an outsider is that the leading models like neuralgcm/weathernext 1 before as well as this model now are all trained with a "crps" objective which I haven't seen at all outside of ml weather prediction. Essentially you add random noise to the inputs and train by minimizing the regular loss (like l1) and at the same time maximizing the difference between…

The random noise is added to the model parameters, not the inputs, or not? This reminds me of variational noise ( https://www.cs.toronto.edu/~graves/nips_2011.pdf ). If it is random noise on the input, it would be like many of the SSL methods, e.g. DINO ( https://arxiv.org/abs/2104.14294 ), right?

Yes you are right it's applied to the parameters, but other models (like ngcm) applied it to the inputs. IMO it shouldn't make a huge difference main point is you max differences between models.

Re: WeatherNext 2: Our most advanced weather forecasting model

#132

Post by lysecret »

counters wrote: post #81
rytill wrote: post #32

Earlier quoted context omitted.

What is the goal of doing that vs using L2 loss?

To add to the existing answers - L2 losses induce a "blurring" effect when you autoregressively roll out these models. That means you not only lose import spatial features, you also truncate the extrema of the predictions - in other terms, you can't forecast high-impact extreme weather with these models at moderate lead times.

Yes very good point this to me is one of the most magical elements of this loss how it suddenly makes the model "collapse" on one output and the predictions become sharp.

Re: WeatherNext 2: Our most advanced weather forecasting model

#133

Post by jachee »

baxtr wrote: post #47
jachee wrote: post #44

Earlier quoted context omitted.

Apple integrated the hyperlocal darksky stuff into their native Weather app. It had a few growing pains, but it's as good as it ever was, imho.

Agreed. The one thing I’d like them to improve are the precipitation maps though. They just feel awkward and unreliable.

Alas, that's just the nature of precipitation maps. When radar is the only true source of what's coming down where, you're a the mercy of the laws of physics, and we don't have a large enough array of stations to give hyperlocal-quality coverage to every locality.

Making it vague-ish was a design choice to help curtail complaints of inaccuracy while still giving near-enough-to-accurate information to be useful generally speaking.

Re: WeatherNext 2: Our most advanced weather forecasting model

#134

Post by NoiseBert69 »

mmooss wrote: post #83
NoiseBert69 wrote: post #6

Earlier quoted context omitted.

As a end user I also want to see the variance to get a feeling of the uncertainty. Quite a lot of weather sites offer this data in an easily eatable visual format.

That would be great - do you recommend any sites?

https://www.meteoblue.com/en/weather/10-days/new-york_united...

https://www.meteoblue.com/en/weather/forecast/multimodelense...

also

https://weather.us/forecast/5128581-new-york/ensemble/euro-r...

Re: WeatherNext 2: Our most advanced weather forecasting model

#135

Post by robotguy »

rkagerer wrote: post #126

I noticed my local weather forecasts from Google search have gotten significantly less accurate these days. Like, they consistenly called for freezing seasonal overnight lows many weeks before it was remotely probable. You'd get better predictions asking anyone who's lived here a couple years. In fairness, I'm in a region that's notoriously difficult to forecast, but the popular non-Google sources seem to be generati…

Not affiliated, just thought it was a neat idea when I stumbled upon it during some hobby weather research last year:

ForecastAdvisor will show you the accuracy of the major weather forecasters, including AerisWeather, Foreca, Microsoft, the National Weather Service, OpenWeather, The Weather Channel, Wetter.com, WeatherBit, World Weather Online, and others. They also provide links to your city's weather forecast from all the other weather forecasters, so you can compare for yourself.

https://www.forecastadvisor.com/

Re: WeatherNext 2: Our most advanced weather forecasting model

#136

Post by counters »

lysecret wrote: post #132
counters wrote: post #81

Earlier quoted context omitted.

To add to the existing answers - L2 losses induce a "blurring" effect when you autoregressively roll out these models. That means you not only lose import spatial features, you also truncate the extrema of the predictions - in other terms, you can't forecast high-impact extreme weather with these models at moderate lead times.

Yes very good point this to me is one of the most magical elements of this loss how it suddenly makes the model "collapse" on one output and the predictions become sharp.

Yeah, it's underplayed in the the writeup here but the context here is important. The "sharpness" issue was a major impediment to improving the skill and utility of these models. When GDM published GenCast two years ago, there was a lot of excitement because the generative approach seemed to completely eliminate this issue. But, there was a trade-off - GenCast was significantly more expensive to train and run inference with, and there wasn't an obvious way to make improvements there. Still faster than an NWP model, but the edge starts to dull.

FGN (and NVIDIA's FourCastNet-v3) show a new path forward that balances inference/training cost without sacrificing the sharpness of the outputs. And you get well-calibrated ensembles if you run them with random seeds to their noise vectors, too!

This is a much bigger deal than people realize.

Re: WeatherNext 2: Our most advanced weather forecasting model

#137

Post by tylervigen »

jasonmarks_ wrote: post #102
tylervigen wrote: post #101

Earlier quoted context omitted.

It's the subtitle of the article they linked to. But to expand: the US flagship forecast model just had its worst year predicting hurricanes since 2005. The trend of errors over the last few years hasn't been great.

More objectively it reads as if none of the models performed well outside of 24 hours with a significant uptick in inaccuracy after 72 hours.

I disagree with your logic. Increased mean error 72 hours out (vs 24 hours out) is not an indication that GFS is getting worse over time. At that scale it’s obviously getting better over time; 24 hours out is further in the future than 72 hours out.

However, an increase in the mean error at the same time out year over year (or between 2005 and 2025) is an indication of an issue, and that’s what we see.

Re: WeatherNext 2: Our most advanced weather forecasting model

#138

Post by rkagerer »

robotguy wrote: post #135
rkagerer wrote: post #126

I noticed my local weather forecasts from Google search have gotten significantly less accurate these days. Like, they consistenly called for freezing seasonal overnight lows many weeks before it was remotely probable. You'd get better predictions asking anyone who's lived here a couple years. In fairness, I'm in a region that's notoriously difficult to forecast, but the popular non-Google sources seem to be generati…

Not affiliated, just thought it was a neat idea when I stumbled upon it during some hobby weather research last year: ForecastAdvisor will show you the accuracy of the major weather forecasters, including AerisWeather, Foreca, Microsoft, the National Weather Service, OpenWeather, The Weather Channel, Wetter.com, WeatherBit, World Weather Online, and others. They also provide links to your city's weather forecast from…

That sounds perfect, I wish it were available outside the US as well.

Re: WeatherNext 2: Our most advanced weather forecasting model

#139

Post by tylervigen »

sweettea wrote: post #66
tylervigen wrote: post #20

Earlier quoted context omitted.

For lay-users they could have explained that better. I think they may not have completely uninformed users in mind for this page though. Developing an ensemble of possible scenarios has been the central insight of weather forecasting since the 1960s when Edward Lorenz discovered that tiny differences in initial conditions can grow exponentially (the "butterfly effect"). Since they could really do it in the 90s, all c…

My understanding is that it's an expected value based on coverage in each of the ensemble scenarios, not quite as simplified as "how many scenarios was there rain in this forecast cell". At least for the US NWS: if 30 of 100 scenarios result in 50% shower coverage, and 70 out of 100 result in 0%, this is reported as 15% chance of rain. Which is exactly the same as 15 with 100% coverage and 85 with 0% coverage, or 100…

Yes, that complexity is what my acknowledgment was meant to refer to.

Re: WeatherNext 2: Our most advanced weather forecasting model

#140

Post by DoctorOetker »

counters wrote: post #79
DoctorOetker wrote: post #60

Earlier quoted context omitted.

Sorry to hijack you: I have some questions regarding current weather models: I am personally not interested in predicting the weather as end users expect it, rather I am interested in representative evolutions of wind patterns. I.e. specify some location (say somewhere in the North Sea, or perhaps on mainland Western Europe), and a date (say Nov 12) without specifying a year, and would like to have the wind patterns…

I mean - you don't need any ML for that. Just go grab random samples from a ~30 day window centered on your day of interest over the region of interest from a reanalysis product like ERA5. If the duration of ERA5 isn't sufficient (e.g. you wouldn't expect on average to see events with a >100 year return period given the limited temporal extent of the dataset) then you could take one step further and pull from an equi…

The ERA5 seems to give hourly data, i.e. nyquist limit would thus give decent oscillation amplitudes for waves with periods of about 5 hours or more, whereas I am more interested in faster timescales seconds, minutes, i.e. wind gusts.

Calculating the stability and structural requirements for a super-chimney to the tropopause, would require representative higher temporal frequency wind fields

Do you know if I can extract such a high time resolution from LENS since a cursory look at ERA5 showed a time resolution of just 1 hour?

The advantage of an ML model is that its usually possible to calculate the joint probability for a wind field, or to selectively generate a dataset with N-th percentile wind fields etc.

If its differentiable, and the structural stress assumptions are known, then one can "optimize" towards wind profiles that are simultaneously more dangerous and more probable, to identify what needs adressing. Thats why an ML model of local wind patterns would be desirable. ML is more than just LLM's. What one typically complains of in the context of LLM's: that there's no error bars on the output, is not entirely correct: just like differentiable ML models for physical and other phenomena they too allow to calculate the joint probability of sentences, except instead of modeling natural phenomena it is modelling what humans uttered in the corpus (or implicit corpus after RLHF etc). A base model LLM can quite accurately predict the likelihood of a human expressing a certain phrase, but thats modeling human expressions, not their validity. An ML model trained on actual weather data, or fine grained simulated weather data results in comparatively more accurate probability distributions, because physics isn't much of an opinion.

Post reply on HN