This needs to be done for more cities in California, especially Los Angeles with its huge geographic area and all the diverse micro climates contained within. Sometimes the weather changes more than 15 degrees in half as many miles from the coast. It therefore doesn't make sense to e.g. check "weather in LA" when its going to be somewhat wrong most of the time depending on where in LA you happen to be, since the litt…
300 meters resolution SF Bay Area Forecast
71–80 of 119 posts
Re: 300 meters resolution SF Bay Area Forecast
#72Earlier quoted context omitted.
Tall bridges do weird things to the wind. I can confirm the bay bridge at surface level, there is functionally no wind for about half a mile downwind from it. Just glassy smooth.
Most of the san mateo bridge is quite low, especially the stretch crossing the bay. This is why I was so shocked it had a wind shadow 10+ miles
Re: 300 meters resolution SF Bay Area Forecast
#73Earlier quoted context omitted.
The website claims to be using DL which may mean less of a model-centric approach? The expertise of the people at the top of the organization, on this problem, seems a little thin, TBH. And, no stated validation results at all? Without such details, this is just marketing. It would be interesting to see how this behaves for longer prediction times and across a range of difficult forcing conditions off the ocean in th…
I agree, this generally left me feeling skeptical. I know of Luca Delle Monache on the advisory team, through colleagues who have researched under him at Scripps and they spoke highly of him. But yes, there is a lot left to the imagination here. With regards to the sfbay specifically I used to work with a fairly high resolution wind model for the bay (this was a more traditional dynamic based simulation) and it worke…
Re: 300 meters resolution SF Bay Area Forecast
#74Earlier quoted context omitted.
I've been working with weather models for 10 years and I often get asked "How accurate is X?" or "Which model is more accurate?" Many people think "accuracy" is a single number or a single thing - it is more complex than this and depends on your needs. This chapter on Numerical Weather Predictions [0] is great, especially the section on "Forecast Quality and Verification" (p777). The eye-opener for me was "Binary/Cat…
Meteoblue was dramatically more accurate in Chamonix last spring than the GFS.
I find global models like GFS are great for understanding the large scale weather systems. The regional high-resolution models, which are usually nested in a global model, give better definition of local weather phenomena like wind shadows or cooler temperatures in valleys.
Dues to averaging, weather simulations usually have a bias error in temperature predictions. These errors are corrected using statistics (look up Model-Output-Statistics) but is hyper-local, i.e., you loose the big picture. This is probably what you're looking at with Meteoblue.
Re: 300 meters resolution SF Bay Area Forecast
#75Earlier quoted context omitted.
I mean up or downsampling is trivial. The question isn't if you can make a raster at any resolution, its if you can make a raster thats accurate and precise at that resolution. Its not clear to me that this is either.
One of the interesting things the model captures at this resolution is the dynamics of the wind going in the bay through the golden gate. See for instance: https://sf.atmo.ai/wind@37.80911,-122.44543,11.68,36,0,16669...
I don't know how the ECMWF model works, but even as someone who did not study meteorology (but studied electrical engineering, which forms the theoretical basis of weather forecasting via the Kalman filter), I can say the following (having spent a number of years working at NCEP): 1. Initial conditions/parameters are fundamental in setting up a model run. 2. Forecasts have for a long time relied on ensembles, which are repeat model runs with slightly varying parameters. The idea of ensembles is, if you run enough of them, you will frequently notice one or more convergence(s) that various sets of parameters produce, e.g. where some sets of parameters predict one movement pattern for a hurricane, while others produce a different movement pattern. Historically, such discrepancies were resolved by actual forecasters, who decided based on their knowledge and experience which one was more likely. In addition, they also had meetings every morning between scientists (developing the model) and forecasters (who relied more on general knowledge and experience) and involved occasionally heated discussions between the groups. But I digress. 3. Considering it involves a chaotic system, I cannot say how much value something like deep learning might bring to the table that produces consistent value above and beyond what's already obtained by using ensembles of Kalman predictive filtering. It is however noteworthy to point out that if the grid resolution is 28,000 meters, then it may not make much sense to set the resolution of the model itself substantially lower (like 300 meters), because any resulting data is more likely to be an artifact of the model itself, rather than reflective of real life information. Luckily, this issue has been and is being addressed through the development of rigorous testing standards, which inform of the inherent quality of forecasts produced by a particular model (this is how they can assign an objective rank to e.g. the GFS and the ECMWF, when forecast quality is generally very close and the model producing the most accurate prediction varies between the two). To put it plainly, the degree to which the website mentioned above has any value is based not on its best predictions, but on the overall variance (i.e. how close predicted data comes to actual measurements of the same, which is necessarily retrospective). 4. That said, it's worthwhile to point out that just because it doesn't involve a government agency with something like a thousand employees, hundreds of scientists (in the case of NCEP alone), and very powerful supercomputers, does not necessarily mean it's bunk (even if it frequently does). For example, I do recall Panasonic (IIRC) showing up out of the blue, with its own forecasting system, which was shown to be competitive after requisite, rigorous testing. I don't remember many details and this was years ago—and its disappearance alone is suspect, but it's worth adding for completeness.
Re: 300 meters resolution SF Bay Area Forecast
#76Growing up in Germany, before I moved to the Bay Area, I was wondering why weather apps and widgets were so prolific. Sure, knowing the forecast for next weekend was nice, but for anything closer I'd just get out of bed and look out of the window. That would pretty much tell me what weather it is, and it would usually change just slowly over a few days or so. Then I moved to the Bay Area, and weather does not only ch…
Another anecdote: In the South of Germany at least, long stretches of sunny days are often followed by sudden thunderstorms with equally sudden bursts of rain. That "fact" had been so deeply ingrained in me that it was subconscious. You'd have a careful feeling if it was hot for too long, suddenly you might find yourself running for the next awning to escape the torrential rain. Sunny weather was a bit like building…
Re: 300 meters resolution SF Bay Area Forecast
#77Growing up in Germany, before I moved to the Bay Area, I was wondering why weather apps and widgets were so prolific. Sure, knowing the forecast for next weekend was nice, but for anything closer I'd just get out of bed and look out of the window. That would pretty much tell me what weather it is, and it would usually change just slowly over a few days or so. Then I moved to the Bay Area, and weather does not only ch…
Yes. I’m from Denmark, but I check the weather every night before stepping out as I have experience 13 C nights where the previous night was 20 C. And this is not uncommon. When I first got here I was stunned by how noticeable nicer the weather was when driving from Santa Clara to Palo Alto, and more than once have I forgotten to bring a sweater to SF.
On the bright side, a hoodie is often all I ever need, all year long.
Re: 300 meters resolution SF Bay Area Forecast
#78Earlier quoted context omitted.
Another anecdote: In the South of Germany at least, long stretches of sunny days are often followed by sudden thunderstorms with equally sudden bursts of rain. That "fact" had been so deeply ingrained in me that it was subconscious. You'd have a careful feeling if it was hot for too long, suddenly you might find yourself running for the next awning to escape the torrential rain. Sunny weather was a bit like building…
I hadn’t even thought of that until I read this. I had thought thunderstorms after hot weather were just a fact of life. I guess in places near German latitudes that get thunderstorms the hot weather is caused by high pressure systems but maybe that isn’t really the cause in California.
Re: 300 meters resolution SF Bay Area Forecast
#79Weather forecasts are so hard for a user to evaluate... Are you going to check it every day and remember how many days it was right or wrong? Please can weather providers just publish a headline statistic of "Our rain/no rain one day ahead forecast is right 85% of the time. That is better than NOAA (80%), Met Office (72%) and weather.com (65%)."
This is kind of the purpose of the "50% chance of rain" things. The process is called calibration and is usually done with linear regression, and it means that in historical forecasts , the actual outcome was rain 50% of the time. Surface precip is notoriously hard to predict, so this is what we've got right now.
Re: 300 meters resolution SF Bay Area Forecast
#80Earlier quoted context omitted.
One of the interesting things the model captures at this resolution is the dynamics of the wind going in the bay through the golden gate. See for instance: https://sf.atmo.ai/wind@37.80911,-122.44543,11.68,36,0,16669...
The Global Forecast System (GFS), i.e. the model presently used at NCEP, has a grid resolution of 18 miles (28 km). It is (has been, for years, actually), the second best global forecast system, right behind the European ECMWF (sometimes outperforming it, but on average slightly underperforming it, in terms of accuracy). I don't know how the ECMWF model works, but even as someone who did not study meteorology (but st…
Kalman filtering is only one part of the process, and plays a critical role during the data assimilation part. Classical Kalman filtering is optimal for Gaussian-distributed linear dynamical systems, but needs tweaks for non Gaussian distributions and non linear systems.
Classical NWP models for instance will integrate the primitive partial differential equations in time and space and run various parameterizations (which can be in some cases even more expensive than integrating the primitive equations). ECMWF on their end use IFS, which is a spectral method for solving the PDEs.
The whole process of solving these models accurately has definitely been some of the most fascinating science and engineering I’ve had the pleasure to work with. It’s extremely humbling :)