Live data from Hacker News

Supercomputer quietly puts U.S. weather resources back on top

usatoday.com

71–79 of 79 posts

Re: Supercomputer quietly puts U.S. weather resources back on top

#71
post #53
post #25

Earlier quoted context omitted.

Smart phones are good at collecting live barometric pressure data but not much else. Temperature sensors aren't very useful because they are poorly calibrated, you can't reliably tell whether someone is outdoors, and the reading can be thrown off by the heat generated by the device or the person holding it. UV sensors are seldom pointed in the right direction. And I haven't seen a phone with a wind speed or precipita…

I genuinely don't mean to sound passive aggressive when I ask, do you know anything about this subject or are you just guessing? I don't know anything about it, but I can think of numerous ways to mitigate the issues you mention. E.g.: * Does it actually matter that the absolute value of temperature sensors is inaccurate, or is it enough that they have good relative accuracy (do they?) * Do you really need to tell if…

I'm not an expert but have played around with sensors on some Android devices. The absolute thermometer accuracy will vary by device. When the temperature rises by 1℃ over the course of an hour is that because it actually got warmer, or is because the user moved inside or put it in a warm jacket pocket or ran another app that caused the CPU to heat up? In theory I suppose if you had thousands of devices in a small area uploading simultaneous temperature readings you could perform a regression analysis and extract a little bit of signal from the noise. But in practice I just don't see how you could ever get enough device density in a small area to make that work. Remember that in urban areas the vast majority of smart phones spend most of their time inside climate-controlled vehicles and buildings, so any temperature readings from those are worthless for weather prediction. But they do still give useful barometric pressure data because the pressure is fairly consistent inside and outside for a given location and altitude.

Re: Supercomputer quietly puts U.S. weather resources back on top

#72
post #69
post #7

Earlier quoted context omitted.

Modern supercomputers essentially are large commodity clusters. Nobody builds a single fast machine ala Cray-1 anymore. It's just not scaleable past a certain point and we're long since past that. However, they still like to give them fancy names as marketing, because they cost tens of millions of dollars. The main differentiation vs a standard datacenter is low-latency high-throughput interconnects, often with a spe…

Does this count, just curious. It is a large single machine not cluster. https://www.sgi.com/pdfs/4555.pdf

A cursory glance at that PDF suggests the exact opposite of your claim.

Re: Supercomputer quietly puts U.S. weather resources back on top

#73

Earlier quoted context omitted.

"Underground" isn't very feasible in Florida. Most of the state is effectively a sandbar with a very high water table. Underground construction is uncommon.

"Underground" isn't very feasible in Florida. Here in central Texas, we have the same thing, but for the opposite reasons. Our water table is pretty deep - typical wells in my town are 900ft deep. But rather than sand, we've got limestone. Typical land has inches of topsoil sitting on top of solid limestone, so excavating a basement is prohibitively expensive.

>Typical land has inches of topsoil sitting on top of solid limestone, so excavating a basement is prohibitively expensive.

So you're telling me one could blast out a cave in the backyard? Spare no expense!

Re: Supercomputer quietly puts U.S. weather resources back on top

#74
post #53
post #25

Earlier quoted context omitted.

Smart phones are good at collecting live barometric pressure data but not much else. Temperature sensors aren't very useful because they are poorly calibrated, you can't reliably tell whether someone is outdoors, and the reading can be thrown off by the heat generated by the device or the person holding it. UV sensors are seldom pointed in the right direction. And I haven't seen a phone with a wind speed or precipita…

I genuinely don't mean to sound passive aggressive when I ask, do you know anything about this subject or are you just guessing? I don't know anything about it, but I can think of numerous ways to mitigate the issues you mention. E.g.: * Does it actually matter that the absolute value of temperature sensors is inaccurate, or is it enough that they have good relative accuracy (do they?) * Do you really need to tell if…

I'm a graduate student in Environmental Fluid Dynamics and I deal with a lot of environmental sensors. The temperature sensors on phones are pretty accurate, but they're quite sensitive to small changes. This means that nradov's original statement is correct: Body heat and cell phone usage are absolutely going to affect the temperature sensor. For example, Sensirion makes a high quality temperature/relative humidity sensor and they recommend the PCB have cutouts around the sensor to offer proper ventilation and minimize heat transfer from surrounding components [1]. On the iPhone 6s, the pressure/temperature sensor is on an edge, but it's also right next to the NAND flash chip, so I'm not quite sure what the heating effects would be.

I can't answer much to your second question but I imagine it's easier in colder environments, especially if the phone is not in a pocket. In addition, here's a publication that claims to be able to get useful signals from noisy data, so I don't know if I would say the measurements are useless [2].

[1] Fig. 10 in https://www.sensirion.com/fileadmin/user_upload/customers/se...

[2] http://onlinelibrary.wiley.com/doi/10.1002/grl.50786/full

Re: Supercomputer quietly puts U.S. weather resources back on top

#75
post #19
post #17

Earlier quoted context omitted.

just curious, do you have references to smartphone sensors being utilized in numerical weather prediction? i'd been browsing through some WMO reports and other associated NWP lit recently and i'd seen stuff on GPS-RO, but nothing on anyone assimilating "smart" devices.

Sure. I've been working on this problem for about 5 years now, starting with the collection of barometric pressure from Android devices. I'm currently working on this with iPhones at Sunshine [1], where we collect pressure data (along with other metrics), but pressure is the most valuable. There are researchers who use this data - we have collected about 4 billion atmospheric pressure measurements that we have distri…

This is really interesting stuff. How are you guys managing that scale of streaming data? Some kind of Kafka based stack?

Re: Supercomputer quietly puts U.S. weather resources back on top

#76
post #69

Earlier quoted context omitted.

Does this count, just curious. It is a large single machine not cluster. https://www.sgi.com/pdfs/4555.pdf

A cursory glance at that PDF suggests the exact opposite of your claim.

I was asking a question, not claiming anything. Thanks for your answer.

Re: Supercomputer quietly puts U.S. weather resources back on top

#78
post #26

Earlier quoted context omitted.

"Underground" isn't very feasible in Florida. Most of the state is effectively a sandbar with a very high water table. Underground construction is uncommon.

And even a slight sea level rise may make large parts of Florida effectively uninhabitable due to repeated flooding. If you want to build a reliable computing facility there then put it on stilts, not underground. http://www.newyorker.com/magazine/2015/12/21/the-siege-of-mi...

Orlando is 82' above sea level.

If you're worried about it flooding, you should probably spend less time building stilts and more time dealing with the fact that Delaware no longer exists, nor does most of New Jersey, New York City, large swathes of the Eastern seaboard, 75% of Louisiana, the Texas coast all the way in to Houston, much of Los Angeles, and all of Sacramento.

On the other hand, it will be nice to splash in the ocean at Joshua Tree National Park, and I'm sure Arkansas residents will be happy to finally be able to own beachfront property.

Re: Supercomputer quietly puts U.S. weather resources back on top

#79
Cliff Mass also blogged [1] about the NOAA supercomputer. While a day after the USA Today's article, many folks appreciate his perspective. He's posted on this several times in the past year [2].

[1] http://cliffmass.blogspot.com/2016/02/the-national-weather-s...

[2] https://www.google.com/webhp?#q=supercomputer+site:+cliffmas...

Post reply on HN