Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NIST was 5 μs off UTC after last week's power cut

jeffgeerling.com

Post reply on HN
Print view View original

91–100 of 162 posts

Re: NIST was 5 μs off UTC after last week's power cut

#91

Post by esseph »

semenko wrote: post #51

I found the most interesting part of the NIST outage post [1] is NIST's special Time Over Fiber (TOF) program [2] that "provides high-precision time transfer by other service arrangements; some direct fiber-optic links were affected and users will be contacted separately." I've never heard of this! Very cool service, presumably for … quant / HFT / finance firms (maybe for compliance with FINRA Rule 4590 [3])? Telecom…

I'm sure all of that is true, but so is "Department of Defense".

They're also the largest holder of IPv4 space, still. https://bgp.he.net/report/peers#_ipv4addresses

Re: NIST was 5 μs off UTC after last week's power cut

#92

Post by secondcoming »

semenko wrote: post #51

I found the most interesting part of the NIST outage post [1] is NIST's special Time Over Fiber (TOF) program [2] that "provides high-precision time transfer by other service arrangements; some direct fiber-optic links were affected and users will be contacted separately." I've never heard of this! Very cool service, presumably for … quant / HFT / finance firms (maybe for compliance with FINRA Rule 4590 [3])? Telecom…

I think Google uses chrony instead of NTP

Re: NIST was 5 μs off UTC after last week's power cut

#93

Post by Denvercoder9 »

herpderperator wrote: post #57
throw0101d wrote: post #35

Earlier quoted context omitted.

> How do those other applications obtain the precise value they need without encountering the Internet issue? They do not use the Internet: they use local (GPS) clocks with internal high-precision clocks for carry-over in case GNSS signal is unavailable: * https://www.ntp.org/support/vendorlinks/ * https://www.meinbergglobal.com/english/products/ntp-time-ser... * https://syncworks.com/shop/syncserver-s650-rubidium-09…

If those other applications use their own local GPS clocks, what is the significance of NIST (and the 5μs inaccuracy) in their scenario?

GPS gets its time from NIST (though during this incident they failed over to another NIST site, so it wasn't impacted).

Re: NIST was 5 μs off UTC after last week's power cut

#94

Post by prpl »

goalieca wrote: post #62
semenko wrote: post #51

I found the most interesting part of the NIST outage post [1] is NIST's special Time Over Fiber (TOF) program [2] that "provides high-precision time transfer by other service arrangements; some direct fiber-optic links were affected and users will be contacted separately." I've never heard of this! Very cool service, presumably for … quant / HFT / finance firms (maybe for compliance with FINRA Rule 4590 [3])? Telecom…

My guess would be scientific experiments where they need to correlate or sequence data over large regions. Things like correlating gravitational waves with radio signals and gamma ray bursts.

those are GPS based too. You typically would have a circuit you trained off off 1PPS and hopefully had a 10 or so satellites in view.

You can get 50ns with this. Of course, you would verify at NIST.

Re: NIST was 5 μs off UTC after last week's power cut

#95

Post by jasonwatkinspdx »

srean wrote: post #55
Topgamer7 wrote: post #44

Out of curiosity, can anyone say the most impactful things they've needed incredibly accurate time for?

Not sure but synthetic massive aperture radio telescope would need syncing their local clocks. I defer to the experts.

As far as I'm aware they just timestamp the sample streams based on a local gps backed atomic reference. Then when they get the data/tapes in one computing center they can just run a more sophisticated correlation entirely in software to smooth things out.

Re: NIST was 5 μs off UTC after last week's power cut

#97

Post by earslap »

V__ wrote: post #10

Has anyone here ever needed microsecond precision? Would love to hear about it.

How do you even get usable microsecond precision sync info from a server thousands of kilometers away? The latency is variable so the information you get can't be verified / will be stale the moment it arrives. I'm quite ignorant on the topic.

Re: NIST was 5 μs off UTC after last week's power cut

#98

Post by throw0101c »

blibble wrote: post #60
dmurray wrote: post #53

Earlier quoted context omitted.

I never saw a need for this in HFT. In my experience, GPS was used instead, but there was never any critical need for microsecond accuracy in live systems. Sub-microsecond latency, yes, but when that mattered it was in order to do something as soon as possible rather than as close as possible to Wall Clock Time X. Still useful for post-trade analysis; perhaps you can determine that a competitor now has a faster conne…

> I never saw a need for this in HFT. In my experience, GPS was used instead, but there was never any critical need for microsecond accuracy in live systems. mifid ii (uk/eu) minimum is 1us granularity https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:...

> mifid ii (uk/eu) minimum is 1us granularity

1us is nothing special for GPS/NTP/PTP appliances (especially with OCXO/rubidium oscillators):

* https://www.microchip.com/en-us/products/clock-and-timing/sy...

* https://www.meinbergglobal.com/english/productinfo/gps-time-...

Re: NIST was 5 μs off UTC after last week's power cut

#99

Post by mcpherrinm »

NetMageSCW wrote: post #77
loph wrote: post #64

Only Boulder servers lost sync. To say NIST was off is clickbait hyperbole. This page: https://tf.nist.gov/tf-cgi/servers.cgi shows that NIST has > 16 NTP servers on IPv4, of those, 5 are in Boulder and were affected by the power failure. The rest were fine. However, most entities should not be using these top-level servers anyway, so this should have been a problem for exactly nobody. IMHO, most applications should…

Who does use those top-level servers? Aren’t some of them propagating the error or are all secondary level servers configured to use dispersed top-level servers? And how do they decide who is right when they don’t match? Is pool.ntp.org dispersed across possible interference and error correlation?

You can look at who the "Stratum 2" servers are, in the NTP.org pool and otherwise. Those are servers who sync from Stratum 1, like NIST.

Anyone can join the NTP.org pool so it's hard to make blanket statements about it. I believe there's some monitoring of servers in the pool but I don't know the details.

For example, Ubuntu systems point to their Stratum 2 timeservers by default, and I'd have to imagine that NIST is probably one of their upstreams.

An NTP server usually has multiple upstream sources and can steer its clock to minimize the error across multiple servers, as well as detecting misbehaving servers and reject them ("Falseticker"). Different NTP server implementations might do this a bit differently.

Re: NIST was 5 μs off UTC after last week's power cut

#100

Post by throw0101c »

herpderperator wrote: post #57
throw0101d wrote: post #35

Earlier quoted context omitted.

> How do those other applications obtain the precise value they need without encountering the Internet issue? They do not use the Internet: they use local (GPS) clocks with internal high-precision clocks for carry-over in case GNSS signal is unavailable: * https://www.ntp.org/support/vendorlinks/ * https://www.meinbergglobal.com/english/products/ntp-time-ser... * https://syncworks.com/shop/syncserver-s650-rubidium-09…

If those other applications use their own local GPS clocks, what is the significance of NIST (and the 5μs inaccuracy) in their scenario?

> If those other applications use their own local GPS clocks, what is the significance of NIST (and the 5μs inaccuracy) in their scenario?

Verification and traceability is one reason: it's all very well to claim you're with-in ±x seconds, but your logs may have to say how close you are to the 'legal reality' that is the official time of NIST.

NIST may also send out time via 'private fibre' for certain purposes:

* https://en.wikipedia.org/wiki/White_Rabbit_Project

'Fibre timing' is also important in case of GNSS signal disruption:

* https://www.gpsworld.com/china-finishing-high-precision-grou...

Post reply on HN