Live data from Hacker News

NTP at NIST Boulder Has Lost Power

lists.nanog.org

121–130 of 218 posts

Re: NTP at NIST Boulder Has Lost Power

#121
post #55

Earlier quoted context omitted.

If your computer was using it as your time server and you didn't have alternatives configured your clock my have drifted a few seconds.

I never checked it, but how much a typical's pc/server's clock does actually drift over a week or a month? I always thought it's well under a second.

I have an extremely cheap and extremely low power automatic cat feed - it’s been on 2 D batteries for 18 months. I just reset it after it had drifted 19 minutes, so about 1 minute a month, or 15 seconds a week!

Re: NTP at NIST Boulder Has Lost Power

#122
post #10

Earlier quoted context omitted.

Time travel is extremely dangerous right now. I highly recommend deferring time travel plans except for extreme temporal emergencies.

Would traveling to the past in order to put in place a preemptive fix for this outage be wise or dangerous? Asking for a friend.

I couldn't comment on the causal hazards but since time is currently having an outage they've got an improved shot at getting away with it. I say go for it.

Re: NTP at NIST Boulder Has Lost Power

#124
post #4

NIST campus status: Due to elevated fire risk and a power outage for the Boulder area, the DOC Boulder Labs campus is CLOSED on December 19 for onsite business and no public access is permitted; previously approved accesses are revoked. [1] WWV still seems to be up, including voice phone access. NIST Boulder has a recorded phone number for site status, and it says that as of December 20, the site is closed with no ac…

Note that WWV is a good 60 miles NNE of Boulder on the outskirts of Fort Collins.

Re: NTP at NIST Boulder Has Lost Power

#125
post #89
post #49

Earlier quoted context omitted.

Maybe this is the disaster plan: There's not a smouldering hole where NIST's Boulder facility used to be, and it will be operational again soon enough. There's no present need for important hard-to-replace sciencey-dudes to go into the shop (which is probably both cold, and dark, and may have other problems that make it unsafe: it's deliberately closed ) to futz around with the the time machines. We still have other…

There are many backup clocks/clusters that NIST uses as redundancies all around Boulder too, no need to even go up to Fort Collins. As in, NIST has fiber to a few at CU and a few commercial companies, last I checked. They're used in cases just like this one. Fun facts about The clock: You can't put anything in the room or take anything out. That's how sensitive the clock is. The room is just filled with asbestos. The…

I'm the Manager of the Computing group at JILA at CU, where utcnist*.colorado.edu used to be housed. Those machines were, for years, consistently the highest bandwidth usage computers on campus.

Unfortunately, the HP cesium clock that backed the utcnist systems failed a few weeks ago, so they're offline. I believe the plan is to decommission those servers anyway - NIST doesn't even list them on the NTP status page anymore, and Judah Levine has retired (though he still comes in frequently). Judah told me in the past that the typical plan in this situation is that you reference a spare HP clock with the clock at NIST, then drive it over to JILA backed by some sort of battery and put it in the rack, then send in the broken one for refurb (~$20k-$40k; new box is closer to $75k). The same is true for the WWVB station, should its clocks fail.

There is fiber that connects NIST to CU (it's part of the BRAN - Boulder Research and Administration Network). Typically that's used when comparing some of the new clocks at JILA (like Jun Ye's strontium clock) to NIST's reference. Fun fact: Some years back the group was noticing loss due to the fiber couplers in various closets between JILA & NIST... so they went to the closets and directly spliced the fibers to each other. It's now one single strand of fiber between JILA & NIST Boulder.

That fiber wasn't connected to the clock that backed utcnist though. utcnist's clock was a commercial cesium clock box from HP that was also fed by GPS. This setup was not particularly sensitive to people being in the room or anything.

Another fun fact: utcnist3 was an FPGA developed in-house to respond to NTP traffic. Super cool project, though I didn't have anything to do with it, haha.

Re: NTP at NIST Boulder Has Lost Power

#126
post #49
post #33

Earlier quoted context omitted.

Somewhat interesting that they themselves don't have access to the site. You'd think there would have been some disaster plans put in place?

Maybe this is the disaster plan: There's not a smouldering hole where NIST's Boulder facility used to be, and it will be operational again soon enough. There's no present need for important hard-to-replace sciencey-dudes to go into the shop (which is probably both cold, and dark, and may have other problems that make it unsafe: it's deliberately closed ) to futz around with the the time machines. We still have other…

> And the orbital GPS clocks are controlled by the US Navy, not NIST.

I thought it was US Space Force / Air Force. Was the Navy previously or currently involved?

Re: NTP at NIST Boulder Has Lost Power

#127
post #10

Earlier quoted context omitted.

Time travel is extremely dangerous right now. I highly recommend deferring time travel plans except for extreme temporal emergencies.

Would traveling to the past in order to put in place a preemptive fix for this outage be wise or dangerous? Asking for a friend.

[deleted]

Re: NTP at NIST Boulder Has Lost Power

#128

One question I have is did DOGE decisions have anything to do with this? Because I know they took knives to NIST.

Residents and some businesses of Boulder have been without power since Tuesday. There was an issue about 10 years ago which caused 1000 homes to burn down and the power company was found liable. They change their actions. Then during the next high wind event, the power company preemptively cut power and businesses sued them for loss of revenue. Now the power company is playing it safe and turning off power to residen…

The Marshall fire was 4 years ago. Almost to the day.

Re: NTP at NIST Boulder Has Lost Power

#130
post #89

Earlier quoted context omitted.

There are many backup clocks/clusters that NIST uses as redundancies all around Boulder too, no need to even go up to Fort Collins. As in, NIST has fiber to a few at CU and a few commercial companies, last I checked. They're used in cases just like this one. Fun facts about The clock: You can't put anything in the room or take anything out. That's how sensitive the clock is. The room is just filled with asbestos. The…

>The actual port for the actual clock, the little metal thingy that is going buzz, buzz, buzz with voltage every second on the dot? Yeah, that little port isn't actually hooked up to anything, as again, it's so sensitive (impedance matching). So they use the other ports on the card for actual data transfer to the rest of the world. Can you restate this part in full technical jargon along with more detail? I'm having…

it sounds like there's some kind of coupling, inductive or optoelectronic

but yes, I also want the juicy details!

so this is the clock

https://en.wikipedia.org/wiki/NIST-F1

or this

https://en.wikipedia.org/wiki/NIST-F2

or there's already F4 too, but it doesn't have a Wikipedia article yet

https://www.nist.gov/news-events/news/2025/04/new-atomic-fou...

but maybe they are talking about the new non-microwave clocks that use Ytterbium-based optical combs ...

or about the Aluminum ion clock

https://www.nist.gov/news-events/news/2025/07/nist-ion-clock...

mind blown

Post reply on HN