Earlier quoted context omitted.
The problem is that we are entering an era of rapid climate shift. Relying primarily on historical data doesn't work that well when you are trying to predict unprecedented weather events. The laws of physics, on the other hand, have not changed - so a simulation will still work just fine.
No we aren't actually. We are still well within historical data, since we use that still to calculate future weather patterns, and there hasn't been anything unprecedented that would trip up a model like this either.
Two new AI-based weather-forecasting systems challenging the status quo
21–30 of 39 posts
Re: Two new AI-based weather-forecasting systems challenging the status quo
#22I've been using meteum.ai from Yandex for a while, in my personal experience it gives the best results.
It looks good, and I know it seems stupid, but without a Fahrenheit setting it isn't useful enough to be used.
Implementing cToF is left as an exercise for the reader
Re: Two new AI-based weather-forecasting systems challenging the status quo
#23I've been using meteum.ai from Yandex for a while, in my personal experience it gives the best results.
It looks good, and I know it seems stupid, but without a Fahrenheit setting it isn't useful enough to be used.
... in the console, and it's almost entirely F. (except for the text summary at the top, which can include other numbers, so isn't safe to mess with this way). You could turn this into a bookmarklet and do this one-click. https://caiorss.github.io/bookmarklet-maker/
I know, totally impractical, no use on a phone, etc. I just wondered how easy it would be to fix.
(edited to add: lol, so mcluck had the same idea. the hour-temp and regex stuff in mine is because replacing just temp__value leaves the hour-by-hour temps in the top panel, which include the degree symbol as part of the text, so I want to leave that alone)
Re: Two new AI-based weather-forecasting systems challenging the status quo
#24Re: Two new AI-based weather-forecasting systems challenging the status quo
#25I remember the good old days when we used to use terms like statistics, heuristics, data science, and machine learning. Now everything is just "AI".
At least these days the LLMs have gotten us a little closer to the original promise of AI.
Re: Two new AI-based weather-forecasting systems challenging the status quo
#26Earlier quoted context omitted.
{In Thick Yorkshire Accent} - 'Luxury! In my day, we used to call it "maths"! Two-hundred and fifty of us huddled together in a bomb shelter in Bracknell[1] poring over oceanographic maps by gaslight and trying not to sneeze into the sergeant's tea.' [1]:https://rafmetman.wordpress.com/meteorology/ww2-meteorology/
Ohhhh 'maths' - we used to DREAM of huddlin' in a shelter doing 'maths'! We barely had a distinction between 'one', 'two' and 'lots'. etc etc
Re: Two new AI-based weather-forecasting systems challenging the status quo
#27Incidentally, this post, along with the rest of the Dark Sky blog, was deleted after they were acquired by Apple.
Re: Two new AI-based weather-forecasting systems challenging the status quo
#28> In the first effort, the team behind Pangu-Weather, trained their system on 39 years of weather data and then asked it to make predictions based on current weather patterns. They found that it was as accurate at doing so as existing systems, and did its work in just a fraction of the time. That's awesome -- years of analysis paying off in a very cool and interesting way. It says that this particular model doesn't p…
Must say that "as accurate at doing so as existing systems" is quite mysterious. Not as accurate as systems from previous decades, not more accurate then the existing systems. What is so special about the now?
This entire class of global numerical weather forecast models has had more-or-less a monotonic increase in forecast accuracy over the past five decades. E.g., a 72 hour forecast from the current generation of these models has similar error statistics to a 24 hour forecast from its predecessors in the early 2000's.
What's special about these AI forecast models is that they are significantly cheaper to run than the existing global numerical weather forecast models. Modern meteorology involves a great deal of statistical analysis to overcome chaotic uncertainty. One way we build these statistical analyses is to run dozens of forecasts with the same model, using slightly different initial conditions, to see how the forecasts diverge. But these ensembles are very under-disperse - a few dozen members just doesn't fully sample the uncertainty. Now, if you can run 1000x the number of ensemble members, a whole new world of possibility opens up.
And that's before you consider just training the AI system to directly output a posterior distribution representing this uncertainty in the first place...
Re: Two new AI-based weather-forecasting systems challenging the status quo
#29> In the first effort, the team behind Pangu-Weather, trained their system on 39 years of weather data and then asked it to make predictions based on current weather patterns. They found that it was as accurate at doing so as existing systems, and did its work in just a fraction of the time. That's awesome -- years of analysis paying off in a very cool and interesting way. It says that this particular model doesn't p…
Must say that "as accurate at doing so as existing systems" is quite mysterious. Not as accurate as systems from previous decades, not more accurate then the existing systems. What is so special about the now?
>> and did its work in just a fraction of the time.
Re: Two new AI-based weather-forecasting systems challenging the status quo
#30I’m actually surprised to hear that the standard approaches used for weather forecasts do not use AI. One would think this would be one of the first areas that would adopt AI given the data available. Also that these new AI systems are performing at the same level of the old non-AI approaches I feel is a real testament to the developers of those systems. This is not something we see in many other cases.
> One would think this would be one of the first areas that would adopt AI given the data available.
Actually, the challenge has been that there _isn't_ enough data available. Sure, we have lots of satellite observations and many other sources, but none of these paint a holistic picture of the atmosphere of the sort you'd need to actually forecast the weather with any precision. ERA-5 - a model-based "re-analysis" that assimilates many observations and tries to create a coherent picture - is only a few years old, and has been the keystone that unlocked all of this development over the past few years.