111–120 of 156 posts
> Astonishingly, this approach, done correctly, produces better forecasts than traditional simulations of the physics of our atmosphere. It seems like this is another instance of The Bitter Lesson, no?
For anyone else who's also in today's lucky 10,000: http://www.incompleteideas.net/IncIdeas/BitterLesson.html
Earlier quoted context omitted.
Maybe between the two of you, you can tell me why my Alexa is telling me there’s no rain today, but it’s raining right now.
You'll need to subscribe to Alexa weather plus, for only 9.99$/month. Now seriously, yes, hyperlocal short-term weather forecast should be a commodity, even public utility?
Earlier quoted context omitted.
No one is claiming that there is "new knowledge" here. The entire class of deep learning or AI-based weather models involves a very specific and simple modeling task. You start with a very large training set which is effectively a historical sequence of "4D pictures" of the atmosphere. Here, "4D" means that you have "pixels" for latitude, longitude, altitude, and time. You have many such pictures of these for relevan…
Thanks for the explanation. I am still a bit confused how this takes care of the errors? I can see how the weather prediction for tomorrow might have less errors. But shouldn't the errors accumulate as you feed the predicted weather as the input for the model? Wouldn't the results start diverging from reality pretty soon? Isn't that the reason why the current limit is close to 6 days? How exactly does this model fixe…
This _class_ of models (not Aurora, or Silurian's model specifically) can potentially improve on this a bit by incorporating forecast error at longer lead times in their core training loss. This is already done in practice for some major models like GraphCast and Stormer. But these models are almost certainly not a magical silver bullet for 10x'ing forecast accuracy.
Earlier quoted context omitted.
Yes, it seems like it. Although I would imagine the features and architecture of the model still take some physics into account. You can't just feed weather data into an LLM, after all.
You'd be surprised :D
It's not exactly an LLM but it works in a similar fashion.
One nit on your framing: NeuralGCM (https://www.nature.com/articles/s41586-024-07744-y), built by my team at Google, is currently at the top of the WeatherBench leaderboard and actually builds in lots of physics :).
We would love to metrics from your model in WeatherBench for comparison. When/if you have that, please do reach out.
Does anyone predict economy/population/... by simulating individual people based on real census information? Monte carlo simulation of major events (births, death, ...) based on known statistics based on age, economic background, location, education, profession, etc.? It seems there are not that many people that this would be computationally infeasible, and states and companies have plenty of data to feed into such s…
This is cool. What do you mean by "defense?"
Earlier quoted context omitted.
You'll need to subscribe to Alexa weather plus, for only 9.99$/month. Now seriously, yes, hyperlocal short-term weather forecast should be a commodity, even public utility?
That makes me appreciate that in Vancouver we have Weatherhood, which is free to use.
Windy(.com) premium also has a great hybrid weather radar+forecast view which was recently released and which I find has been very effective at predicting rain at a specific location on the map vs "nearby". With smaller weather patterns it is entirely possible for it to rain a few blocks away but not at your location. An 11-KM resolution weather forecast (as referenced above) will not be able to capture this nuance.
Earlier quoted context omitted.
Low-income neighborhoods are good signal indicator for flooding high risk zones. There's a demographic angle, too.
Are you suggesting that flood prevention only happens in higher income neighbourhoods? Flood prevention tends to lie on the county engineers. Not so much private individuals to dictate. Doesn't matter how much money you have, you can't just dig up a road to put in proper flood prevention measures like drainage and grade.
Glad to see that you can make ensemble forecasts of tropical cyclones! This absolutely essential for useful weather forecasts of uncertain events, and I am a little dissapointed by the frequent comparisons (not just you) of ML models to ECMWF's deterministic HRES model. HRES is more of a single realization of plausible weather, rather than an best estimate of "average" weather, so this is a bit of apples vs oranges.…
Re NeuralGCM, indeed, our post should have said "*most* of these models". Definitely proves that combining ML and physics models can work really well. Thanks for your comments!