Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Supercomputer quietly puts U.S. weather resources back on top

usatoday.com

Post reply on HN
Print view View original

21–30 of 79 posts

Re: Supercomputer quietly puts U.S. weather resources back on top

#21

Post by munificent »

nxzero wrote: post #8

Maybe it's me, but having the backup in Orlando sounds like a bad idea. Anyone able to comment on the logic behind this?

Florida, for obvious reasons, cares a lot about weather forecasting and is likely more willing to pony up some of the costs, infrastructure, staff, etc. to support this.

Meanwhile, Orlando itself is relatively safe when it comes to hurricanes. Few make it far enough inland to cause significant damage there. (Though, obviously, some do. I was there during the 2004 season.)

Re: Supercomputer quietly puts U.S. weather resources back on top

#22

Post by mrow84 »

mturmon wrote: post #12
matt2000 wrote: post #2

Is there anyone here who works on weather models who would be able to explain why custom supercomputers are still needed vs. large commodity clusters? Thanks!

I think the comments nearby about high communications bandwidth required by dense systems of equations are true -- basically they distinguish a tuned, integrated system sold by a vendor like Cray from a row of racks with commodity systems. The point I wanted to add is that ECMWF, generally acknowledged to be the leader in NWP, also uses a pair of large Cray supercomputers ( http://www.ecmwf.int/en/computing/our-facil…

Yes, it is the interconnects that are important. The UK Met Office has also recently upgraded to a Cray XC40 [0], and there are other XC30 systems in the UK NSS [1] and the US NERSC [2], both of which are used for weather/climate models, amongst other things.

[0] http://www.metoffice.gov.uk/news/releases/archive/2014/new-h...

[1] http://www.archer.ac.uk/about-archer/

[2] http://www.nersc.gov/systems/edison-cray-xc30/

Re: Supercomputer quietly puts U.S. weather resources back on top

#23

Post by munificent »

chm wrote: post #20
nxzero wrote: post #8

Maybe it's me, but having the backup in Orlando sounds like a bad idea. Anyone able to comment on the logic behind this?

Could they have built it underground?

"Underground" isn't very feasible in Florida. Most of the state is effectively a sandbar with a very high water table. Underground construction is uncommon.

Re: Supercomputer quietly puts U.S. weather resources back on top

#24

Post by privong »

dhsjhdjs wrote: post #9

For a newbie in this domain, how do we start doing our own toy forecasts (lets say for Bay area)?

> For a newbie in this domain, how do we start doing our own toy forecasts (lets say for Bay area)?

There are some available weather modeling packages, such as WRF[0], as a mesoscale (few km to 1000 kms) model, that one can download and run. You'd obviously need to find a resource for the appropriate input data. I don't know how easy it is to get up and running, though.

[0] http://www.wrf-model.org/index.php

Re: Supercomputer quietly puts U.S. weather resources back on top

#25

Post by nradov »

cryptoz wrote: post #16
reubenmorais wrote: post #13

Earlier quoted context omitted.

> Think about every single iPhone and every single Galaxy S in the world as a useful sensor array _and_ a parallel processor... Sadly, until we have a breakthrough in battery technology, all of those will be parallel processors programmed to be asleep as often as they can.

Sure, that's true - but they can just as happily run their GPUs if they are plugged in and charging without any battery issues. Although for the moment you are right, they aren't that useful as a powerful distributed supercomputer. They ARE, however, really really good at collecting live weather data and sending that to the supercomputer to process. One step at a time.

Smart phones are good at collecting live barometric pressure data but not much else. Temperature sensors aren't very useful because they are poorly calibrated, you can't reliably tell whether someone is outdoors, and the reading can be thrown off by the heat generated by the device or the person holding it. UV sensors are seldom pointed in the right direction. And I haven't seen a phone with a wind speed or precipitation sensor yet.

Re: Supercomputer quietly puts U.S. weather resources back on top

#26

Post by nradov »

munificent wrote: post #23
chm wrote: post #20

Earlier quoted context omitted.

Could they have built it underground?

"Underground" isn't very feasible in Florida. Most of the state is effectively a sandbar with a very high water table. Underground construction is uncommon.

And even a slight sea level rise may make large parts of Florida effectively uninhabitable due to repeated flooding. If you want to build a reliable computing facility there then put it on stilts, not underground.

http://www.newyorker.com/magazine/2015/12/21/the-siege-of-mi...

Re: Supercomputer quietly puts U.S. weather resources back on top

#28

Post by ThomPete »

cryptoz wrote: post #3

It's a very exciting time in weather forecasting. We're getting all kinds of new observations available, from cheap LEO satellites to smartphone sensors to smart umbrellas and everything else. And faster computers! And more parallel computers! Think about every single iPhone and every single Galaxy S in the world as a useful sensor array _and_ a parallel processor... Weather forecasting accuracy is still constrained…

In other words. This is how the actual IoT revolution (if one can call it that) looks like.

Re: Supercomputer quietly puts U.S. weather resources back on top

#29

Post by kkylin »

acd wrote: post #11

Great that there is a more powerful computer helping predict the weather! One should know that weather is a chaotic system and after three days its very hard to predict the weather correctly. This is known by the lorenz equations. http://www.uvm.edu/~cdanfort/research/danforth-bates-thesis.... Here is the UK met office. "Most of the time the atmosphere behaves rather like the lower-left picture where we can predict w…

Indeed! though the constraints that chaos imposes on predictability is a subtle question. See, e.g., ftp://mana.soest.hawaii.edu/pub/rlukas/LSASI/ENSO/Prediction/Predicability_a_Problem_2006.pdf

Re: Supercomputer quietly puts U.S. weather resources back on top

#30

Post by kuya »

cryptoz wrote: post #19
kuya wrote: post #17

Earlier quoted context omitted.

just curious, do you have references to smartphone sensors being utilized in numerical weather prediction? i'd been browsing through some WMO reports and other associated NWP lit recently and i'd seen stuff on GPS-RO, but nothing on anyone assimilating "smart" devices.

Sure. I've been working on this problem for about 5 years now, starting with the collection of barometric pressure from Android devices. I'm currently working on this with iPhones at Sunshine [1], where we collect pressure data (along with other metrics), but pressure is the most valuable. There are researchers who use this data - we have collected about 4 billion atmospheric pressure measurements that we have distri…

ah, thanks, that's fascinating.

"It also learns every time you actively report to the community on sky conditions and hazards, translating this information into weather predictions."

does sunshine actively run a numerical weather prediction model like WRF?

it'd be great to assimilate all these extra sensors into the NWP centre's models, but i imagine things like cal/val and WMO agreements to share data might make things difficult for commercial companies?

Post reply on HN