Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NTP at NIST Boulder Has Lost Power

lists.nanog.org

Post reply on HN
Print view View original

51–60 of 218 posts

Re: NTP at NIST Boulder Has Lost Power

#51

Post by xorcist »

puzzlingcaptcha wrote: post #34
guenthert wrote: post #29

Earlier quoted context omitted.

Ubuntu using atomic clocks would surprise me. Sure they could, but it's not obvious to me why they would spend $$$$ on such. More plausible to me seems that they would be using GPSDO as reference clocks (in this context, about as good as your own atomic clock), iff they were running their own time servers. Google finds only that they are using servers from the NTP Pool Project, which will be using a variety of refere…

I think people have a wrong idea of what a modern atomic clock looks like. These are readily available commercially, Microchip for example will happily sell you hydrogen, cesium or rubidium atomic clocks. Hydrogen masers are rather unwieldy, but you can get a rubidium clock in a 1U format and cesium ones are not much bigger. I think their cesium freq standards are formerly a HP business they acquired. Example: https:…

It is also important to realize that an atomic clock will only give you a steady pulse. It will count seconds for you, and do so very accurately, but that is not the same as knowing what time it is.

If you get a rubidium clock for your garage, you can sync it up with GPS to get an accurate-enough clock for your hobby NTP project, but large research institutions and their expensive contraptions are more elaborate to set up.

Re: NTP at NIST Boulder Has Lost Power

#52

Post by jhart99 »

cdfuller wrote: post #5

Can anybody expand on the implications of this? Being unfamiliar with it, it's hard to tell if this is a minor blip that happens all the time, or if it's potentially a major issue that could cause cascading errors equal to the hype of Y2K.

NIST maintains several time standards. Gaithersburg MD is still up and I assume Hawaii is as well. Other than potential damage to equipment from loss of power (turbo molecular vacuum pumps and oil diffusion pumps might end up failing in interesting ways if not shut down properly) it will just take some time for the clocks to be recalibrated against the other NIST standards.

Re: NTP at NIST Boulder Has Lost Power

#53

Post by sdrmill »

lovich wrote: post #48
Maxious wrote: post #41

Earlier quoted context omitted.

There's two other sites for the time.nist.gov service so it'll be okay. Probably more interesting is how you get a tier 0 site back in sync - NIST rents out these cyberpunk looking units you can use to get your local frequency standards up to scratch for ~$700/month https://www.nist.gov/programs-projects/frequency-measurement...

What happens in the event all the sites for time.nist.gov go down? is it included in the spec? Also thank you for that link, this is exactly the kind of esoteric knowledge that I enjoy learning about

Most high-availability networks use pool.ntp.org or vendor-specific pools (e.g., time.cloudflare.com, time.google.com, time.windows.com). These systems would automatically switch to a surviving peer in the pool.

Many data centers and telecom hubs use local GPS/GNSS-disciplined oscillators or atomic clocks and wouldn’t be affected.

Most laptops, smartphones, tablets, etc. would be accurate enough for days before drift affected things for the most part.

Kerberos requires clocks to be typically within 5 minutes to prevent replay attacks, so they’d probably be ok.

Sysadmins would need to update hardcoded NTP configurations to point to secondary servers.

If timestamps were REALLY off, TLS certificates might fail, but that’s highly unlikely.

Databases could be corrupted due to failure of transaction ordering.

Financial exchanges are often legally required to use time traceable to a national standard like UTC(NIST). A total failure of the NIST distribution layer could potentially trigger a suspension of electronic trading to maintain audit trail integrity.

Modern power grids use Synchrophasors that require microsecond-level precision for frequency monitoring. Losing the NIST reference would degrade the grid's ability to respond to load fluctuations, increasing the risk of cascading outages.

Re: NTP at NIST Boulder Has Lost Power

#54

Post by ExoticPearTree »

jeffrallen wrote: post #42
autarch wrote: post #10

Earlier quoted context omitted.

Time travel is extremely dangerous right now. I highly recommend deferring time travel plans except for extreme temporal emergencies.

Would traveling to the past in order to put in place a preemptive fix for this outage be wise or dangerous? Asking for a friend.

Tell your friend that this course of action failed, as us in the present are still experiencing issues.

Re: NTP at NIST Boulder Has Lost Power

#55

Post by Roark66 »

ThrowawayTestr wrote: post #11
cdfuller wrote: post #5

Can anybody expand on the implications of this? Being unfamiliar with it, it's hard to tell if this is a minor blip that happens all the time, or if it's potentially a major issue that could cause cascading errors equal to the hype of Y2K.

If your computer was using it as your time server and you didn't have alternatives configured your clock my have drifted a few seconds.

I never checked it, but how much a typical's pc/server's clock does actually drift over a week or a month? I always thought it's well under a second.

Re: NTP at NIST Boulder Has Lost Power

#56

Post by Vosporos »

amelius wrote: post #45
TylerE wrote: post #37

Earlier quoted context omitted.

Step One of most disaster plans is not to create a second emergency.

But can't NTP server downtime cause a disaster?

One (amongst many) NTP server going down creates less issues than an NTP server spreading wrong time.

Re: NTP at NIST Boulder Has Lost Power

#57

Post by macintux »

Vosporos wrote: post #56
amelius wrote: post #45

Earlier quoted context omitted.

But can't NTP server downtime cause a disaster?

One (amongst many) NTP server going down creates less issues than an NTP server spreading wrong time.

General rule of thumb: a misbehaving/slow server in any well-architected distributed system is vastly worse than a dead server.

Re: NTP at NIST Boulder Has Lost Power

#58

Post by autarch »

ExoticPearTree wrote: post #54
jeffrallen wrote: post #42

Earlier quoted context omitted.

Would traveling to the past in order to put in place a preemptive fix for this outage be wise or dangerous? Asking for a friend.

Tell your friend that this course of action failed, as us in the present are still experiencing issues.

Well, that's _this_ timeline. Other timelines never had an outage.

Re: NTP at NIST Boulder Has Lost Power

#59

Post by varjag »

amelius wrote: post #47

This makes me wonder, if you take the average time of all wristwatches on the planet, accounting for timezones and throwing out outliers, how close would you get to NTP time? And how many randomly chosen wristwatches would you need to get anything reasonable?

Close but unlikely to be precise in metrology sense. There's unlikely even a billion wrist watches being worn.

Re: NTP at NIST Boulder Has Lost Power

#60

Post by b112 »

amelius wrote: post #47

This makes me wonder, if you take the average time of all wristwatches on the planet, accounting for timezones and throwing out outliers, how close would you get to NTP time? And how many randomly chosen wristwatches would you need to get anything reasonable?

One. One watch. POTUS's watch. And in fact, that's why Boulder is currently shuttered... they disagreed.
Post reply on HN