Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Supercomputer quietly puts U.S. weather resources back on top

usatoday.com

Post reply on HN
Print view View original

11–20 of 79 posts

Re: Supercomputer quietly puts U.S. weather resources back on top

#11

Post by acd »

Great that there is a more powerful computer helping predict the weather! One should know that weather is a chaotic system and after three days its very hard to predict the weather correctly. This is known by the lorenz equations.

http://www.uvm.edu/~cdanfort/research/danforth-bates-thesis....

Here is the UK met office. "Most of the time the atmosphere behaves rather like the lower-left picture where we can predict with confidence for a few days and have to use probabilities thereafter. " http://research.metoffice.gov.uk/research/nwp/ensemble/conce...

Re: Supercomputer quietly puts U.S. weather resources back on top

#12

Post by mturmon »

matt2000 wrote: post #2

Is there anyone here who works on weather models who would be able to explain why custom supercomputers are still needed vs. large commodity clusters? Thanks!

I think the comments nearby about high communications bandwidth required by dense systems of equations are true -- basically they distinguish a tuned, integrated system sold by a vendor like Cray from a row of racks with commodity systems.

The point I wanted to add is that ECMWF, generally acknowledged to be the leader in NWP, also uses a pair of large Cray supercomputers (http://www.ecmwf.int/en/computing/our-facilities/supercomput...).

Note that the Cray XC30 uses Xeon processors and a custom interconnect. The old ECMWF system used IBM Power chips.

Re: Supercomputer quietly puts U.S. weather resources back on top

#13

Post by reubenmorais »

cryptoz wrote: post #3

It's a very exciting time in weather forecasting. We're getting all kinds of new observations available, from cheap LEO satellites to smartphone sensors to smart umbrellas and everything else. And faster computers! And more parallel computers! Think about every single iPhone and every single Galaxy S in the world as a useful sensor array _and_ a parallel processor... Weather forecasting accuracy is still constrained…

> Think about every single iPhone and every single Galaxy S in the world as a useful sensor array _and_ a parallel processor...

Sadly, until we have a breakthrough in battery technology, all of those will be parallel processors programmed to be asleep as often as they can.

Re: Supercomputer quietly puts U.S. weather resources back on top

#15

Post by superuser2 »

jordache wrote: post #14

Wish they would stop using the library of congress' collection as a reference... I know it's big.. no idea how big.. it's not tangible to me.. never visited.. I'm sure most americans have not visited.

You do not get an idea of how big it is by visiting; the stacks are off limits and much of the material must be ordered in from suburban warehouses.

Re: Supercomputer quietly puts U.S. weather resources back on top

#16

Post by cryptoz »

reubenmorais wrote: post #13
cryptoz wrote: post #3

It's a very exciting time in weather forecasting. We're getting all kinds of new observations available, from cheap LEO satellites to smartphone sensors to smart umbrellas and everything else. And faster computers! And more parallel computers! Think about every single iPhone and every single Galaxy S in the world as a useful sensor array _and_ a parallel processor... Weather forecasting accuracy is still constrained…

> Think about every single iPhone and every single Galaxy S in the world as a useful sensor array _and_ a parallel processor... Sadly, until we have a breakthrough in battery technology, all of those will be parallel processors programmed to be asleep as often as they can.

Sure, that's true - but they can just as happily run their GPUs if they are plugged in and charging without any battery issues. Although for the moment you are right, they aren't that useful as a powerful distributed supercomputer. They ARE, however, really really good at collecting live weather data and sending that to the supercomputer to process. One step at a time.

Re: Supercomputer quietly puts U.S. weather resources back on top

#17

Post by kuya »

cryptoz wrote: post #3

It's a very exciting time in weather forecasting. We're getting all kinds of new observations available, from cheap LEO satellites to smartphone sensors to smart umbrellas and everything else. And faster computers! And more parallel computers! Think about every single iPhone and every single Galaxy S in the world as a useful sensor array _and_ a parallel processor... Weather forecasting accuracy is still constrained…

just curious, do you have references to smartphone sensors being utilized in numerical weather prediction?

i'd been browsing through some WMO reports and other associated NWP lit recently and i'd seen stuff on GPS-RO, but nothing on anyone assimilating "smart" devices.

Re: Supercomputer quietly puts U.S. weather resources back on top

#18

Post by tyingq »

jordache wrote: post #14

Wish they would stop using the library of congress' collection as a reference... I know it's big.. no idea how big.. it's not tangible to me.. never visited.. I'm sure most americans have not visited.

This helped a little:

"The Library of Congress is the largest library in the world, with more than 162 million items on approximately 838 miles of bookshelves. The collections include more than 38 million books and other print materials, 3.6 million recordings, 14 million photographs, 5.5 million maps, 7.1 million pieces of sheet music and 70 million manuscripts."[1]

[1] https://www.loc.gov/about/fascinating-facts/

Re: Supercomputer quietly puts U.S. weather resources back on top

#19

Post by cryptoz »

kuya wrote: post #17
cryptoz wrote: post #3

It's a very exciting time in weather forecasting. We're getting all kinds of new observations available, from cheap LEO satellites to smartphone sensors to smart umbrellas and everything else. And faster computers! And more parallel computers! Think about every single iPhone and every single Galaxy S in the world as a useful sensor array _and_ a parallel processor... Weather forecasting accuracy is still constrained…

just curious, do you have references to smartphone sensors being utilized in numerical weather prediction? i'd been browsing through some WMO reports and other associated NWP lit recently and i'd seen stuff on GPS-RO, but nothing on anyone assimilating "smart" devices.

Sure. I've been working on this problem for about 5 years now, starting with the collection of barometric pressure from Android devices. I'm currently working on this with iPhones at Sunshine [1], where we collect pressure data (along with other metrics), but pressure is the most valuable.

There are researchers who use this data - we have collected about 4 billion atmospheric pressure measurements that we have distributed for academic and government research. The primary researchers are Cliff Mass and his lab at the University of Washington. There are also groups in Canada and the US that are using the data. IBM is now also collecting and using smartphone pressure data through their mobile apps. [2]

Generally speaking, the current trend is to take the live data stream, run it through a quality-control algorithm and then use kalman filters in the WRF data assimilation package.

There are some papers published, but it is still early. I will find some links to papers if you'd like to read them. [3]

[1] https://thesunshine.co/

[2] http://www.nytimes.com/2015/10/29/technology/ibm-to-acquire-...

[3] Utility of Dense Pressure Observations for Improving Mesoscale Analyses and Forecasts: http://www.atmos.washington.edu/~hakim/papers/madaus_hakim_m...

Post reply on HN