Live data from Hacker News

Two new AI-based weather-forecasting systems challenging the status quo

phys.org

21–30 of 39 posts

Re: Two new AI-based weather-forecasting systems challenging the status quo

#21
post #13

Earlier quoted context omitted.

The problem is that we are entering an era of rapid climate shift. Relying primarily on historical data doesn't work that well when you are trying to predict unprecedented weather events. The laws of physics, on the other hand, have not changed - so a simulation will still work just fine.

No we aren't actually. We are still well within historical data, since we use that still to calculate future weather patterns, and there hasn't been anything unprecedented that would trip up a model like this either.

So global warming is just a hoax? Or what are you saying?

Re: Two new AI-based weather-forecasting systems challenging the status quo

#22
post #6

I've been using meteum.ai from Yandex for a while, in my personal experience it gives the best results.

It looks good, and I know it seems stupid, but without a Fahrenheit setting it isn't useful enough to be used.

Array.from(document.querySelectorAll('.temp__value')).map(el => el.innerText = Math.floor(cToF(Number(el.innerText))))

Implementing cToF is left as an exercise for the reader

Re: Two new AI-based weather-forecasting systems challenging the status quo

#23
post #6

I've been using meteum.ai from Yandex for a while, in my personal experience it gives the best results.

It looks good, and I know it seems stupid, but without a Fahrenheit setting it isn't useful enough to be used.

Array.from(document.querySelectorAll(".temp__value,.fact__hour-temp")).forEach(e=>e.textContent=e.textContent.replace(/[-+]?[0-9.]+/g, t=>(t*9/5)+32))

... in the console, and it's almost entirely F. (except for the text summary at the top, which can include other numbers, so isn't safe to mess with this way). You could turn this into a bookmarklet and do this one-click. https://caiorss.github.io/bookmarklet-maker/

I know, totally impractical, no use on a phone, etc. I just wondered how easy it would be to fix.

(edited to add: lol, so mcluck had the same idea. the hour-temp and regex stuff in mine is because replacing just temp__value leaves the hour-by-hour temps in the top panel, which include the degree symbol as part of the text, so I want to leave that alone)

Re: Two new AI-based weather-forecasting systems challenging the status quo

#25

I remember the good old days when we used to use terms like statistics, heuristics, data science, and machine learning. Now everything is just "AI".

Eh, "AI" has always (at least a decade+) been thrown around as a catch all term for things that should have been called ML or DS or statistics (though I find that term a bit reductive, it's like saying it's all just math).

At least these days the LLMs have gotten us a little closer to the original promise of AI.

Re: Two new AI-based weather-forecasting systems challenging the status quo

#26

Earlier quoted context omitted.

{In Thick Yorkshire Accent} - 'Luxury! In my day, we used to call it "maths"! Two-hundred and fifty of us huddled together in a bomb shelter in Bracknell[1] poring over oceanographic maps by gaslight and trying not to sneeze into the sergeant's tea.' [1]:https://rafmetman.wordpress.com/meteorology/ww2-meteorology/

Ohhhh 'maths' - we used to DREAM of huddlin' in a shelter doing 'maths'! We barely had a distinction between 'one', 'two' and 'lots'. etc etc

You were lucky to have ones and twos! The best we could manage was to rifle through tea leaves at the bottom of a cracked cup to divine the forecast for the next six hours in Thebes. And when we would read the leaves wrong, our dad would thrash us in two with bronze bread knife and dance about our graves singing Ode to Dionysus.

Re: Two new AI-based weather-forecasting systems challenging the status quo

#27
Old (from 2011) but Dark Sky had a blog post on how local, short-term weather prediction works: https://web.archive.org/web/20200122042359/https://blog.dark...

Incidentally, this post, along with the rest of the Dark Sky blog, was deleted after they were acquired by Apple.

Re: Two new AI-based weather-forecasting systems challenging the status quo

#28
post #18

> In the first effort, the team behind Pangu-Weather, trained their system on 39 years of weather data and then asked it to make predictions based on current weather patterns. They found that it was as accurate at doing so as existing systems, and did its work in just a fraction of the time. That's awesome -- years of analysis paying off in a very cool and interesting way. It says that this particular model doesn't p…

Must say that "as accurate at doing so as existing systems" is quite mysterious. Not as accurate as systems from previous decades, not more accurate then the existing systems. What is so special about the now?

It's specifically referring to these forecast models having accuracy about on par with the state-of-the-science global numerical weather forecast models. "Accuracy" here specifically means esoteric metrics like the "500mb anomaly correlation coefficient" (basically a summary statistic that tells you how well the 3D atmosphere fields predicted by the model match what we observe later on).

This entire class of global numerical weather forecast models has had more-or-less a monotonic increase in forecast accuracy over the past five decades. E.g., a 72 hour forecast from the current generation of these models has similar error statistics to a 24 hour forecast from its predecessors in the early 2000's.

What's special about these AI forecast models is that they are significantly cheaper to run than the existing global numerical weather forecast models. Modern meteorology involves a great deal of statistical analysis to overcome chaotic uncertainty. One way we build these statistical analyses is to run dozens of forecasts with the same model, using slightly different initial conditions, to see how the forecasts diverge. But these ensembles are very under-disperse - a few dozen members just doesn't fully sample the uncertainty. Now, if you can run 1000x the number of ensemble members, a whole new world of possibility opens up.

And that's before you consider just training the AI system to directly output a posterior distribution representing this uncertainty in the first place...

Re: Two new AI-based weather-forecasting systems challenging the status quo

#29
post #18

> In the first effort, the team behind Pangu-Weather, trained their system on 39 years of weather data and then asked it to make predictions based on current weather patterns. They found that it was as accurate at doing so as existing systems, and did its work in just a fraction of the time. That's awesome -- years of analysis paying off in a very cool and interesting way. It says that this particular model doesn't p…

Must say that "as accurate at doing so as existing systems" is quite mysterious. Not as accurate as systems from previous decades, not more accurate then the existing systems. What is so special about the now?

>What is so special about the now

>> and did its work in just a fraction of the time.

Re: Two new AI-based weather-forecasting systems challenging the status quo

#30
post #4

I’m actually surprised to hear that the standard approaches used for weather forecasts do not use AI. One would think this would be one of the first areas that would adopt AI given the data available. Also that these new AI systems are performing at the same level of the old non-AI approaches I feel is a real testament to the developers of those systems. This is not something we see in many other cases.

ML/AI has been used in the weather forecasting world since the 1970's - most often to post-process forecast model output to better calibrate it against observations and correct for biases.

> One would think this would be one of the first areas that would adopt AI given the data available.

Actually, the challenge has been that there _isn't_ enough data available. Sure, we have lots of satellite observations and many other sources, but none of these paint a holistic picture of the atmosphere of the sort you'd need to actually forecast the weather with any precision. ERA-5 - a model-based "re-analysis" that assimilates many observations and tries to create a coherent picture - is only a few years old, and has been the keystone that unlocked all of this development over the past few years.

Post reply on HN