It’s crazy we’ll never have forecasts as good as dark sky again. What kind of magic were they doing?
'Nowcasting' is an area of active research, both with machine learning and with physics-informed or visual flow approaches.
Part of the problem from the machine learning side is that these are _huge_ problems. NVidia's StormCast (https://research.nvidia.com/publication/2024-08_kilometer-sc...) works globally at kilometer scales, and you can imagine how big those grids are. Even with patch training, you're dealing with very large datasets.
At the same time, this is not exactly a high-profile area of research. National weather centres focus on actionable medium-range weather predictions, and meteorologists can look at radar themselves and perform mark-one-eyeball predictions for very short-range watches and warnings. Some private-sector actors will pay for short range predictions, but they're often looking for something hyperlocalized (e.g. weather at this particular construction site, for crane safety) or specialized (near-real-time cloud and wind predictions for renewable energy).
Most public-accessible weather predictions are downstream of either a public-sector effort (which doesn't internalize benefits, leading to under-resourcing) or a byproduct of another private-sector offering.