For anyone else wondering how their clients verify the authenticity of timestamps; It seems they use the NTP server software Chrony, which has support for NTS (Network Time Security, RFC 8915).
Open-sourcing a more precise time appliance
21–30 of 188 posts
Re: Open-sourcing a more precise time appliance
#22> While building our Time Appliance, we also invented a Time Card, a PCIe card that can turn any commodity server into a time appliance.
Invented? These cards have been around. e.g. Meinberg makes a variety of expansion cards which use various time keeping sources such as WWVB, IRIG-B, GNSS, etc.
Re: Open-sourcing a more precise time appliance
#23I'm still unclear why they need the MAC on top of the GNSS reception. I can understand if it's just to provide holdover in event of GNSS signal loss, but they seem to be saying that it provides better precision when you've got both.
If I read correctly: GNSS PPS is more jittery but does not drift. The MAC has 1000x less jitter but drifts. You can cleverly combine them with statistics and magic to get the best of both worlds.
From what I remember of textbook discussions of atomic clocks, stabilizing drifting oscillators is also in part how normal, old (50+ years) atomic clocks work as well, isn’t it? You have a fiddly and unreasonably high-frequency reference, you slave a relatively garden-variety standard like an environmentally controlled piezoelectric crystal to it, and then feed your periodic signals, counters, etc. off that.
Re: Open-sourcing a more precise time appliance
#24I'm still unclear why they need the MAC on top of the GNSS reception. I can understand if it's just to provide holdover in event of GNSS signal loss, but they seem to be saying that it provides better precision when you've got both.
I think they mean better precision during holdover because they model any inaccuracy of the rubidium clock while GNSS is up in order to compensate the local clock when GNSS isn't working.
Re: Open-sourcing a more precise time appliance
#25I'm still unclear why they need the MAC on top of the GNSS reception. I can understand if it's just to provide holdover in event of GNSS signal loss, but they seem to be saying that it provides better precision when you've got both.
If I read correctly: GNSS PPS is more jittery but does not drift. The MAC has 1000x less jitter but drifts. You can cleverly combine them with statistics and magic to get the best of both worlds.
Phase-locked loops[0] are not magic ;)
Also I imagine they are using a bog standard Kalman filter[1].
Re: Open-sourcing a more precise time appliance
#26Earlier quoted context omitted.
If I read correctly: GNSS PPS is more jittery but does not drift. The MAC has 1000x less jitter but drifts. You can cleverly combine them with statistics and magic to get the best of both worlds.
From what I remember of textbook discussions of atomic clocks, stabilizing drifting oscillators is also in part how normal, old (50+ years) atomic clocks work as well, isn’t it? You have a fiddly and unreasonably high-frequency reference, you slave a relatively garden-variety standard like an environmentally controlled piezoelectric crystal to it, and then feed your periodic signals, counters, etc. off that.
Yes. Surprisingly, atomic clocks have high jitter compared to ovenized quartz oscillators. Every atomic clock package is outputting its signal from a quartz oscillator that is disciplined by the atomic oscillator.
Re: Open-sourcing a more precise time appliance
#27Why does Facebook need a precision time server?
There are probably a few reasons. One that I can imagine is that they use a distributed database somewhere that benefits from reliably tight wall clocks (in the style of Cloud Spanner or Cockroach).
Cockroach is much less dependent on highly accurate clocks. That was basically the core goal. Spanner like, without expensive clock infrastructure. Though it sounds like CockroachDB does better with at least reasonably well synced clocks.
https://www.cockroachlabs.com/blog/living-without-atomic-clo...
Re: Open-sourcing a more precise time appliance
#28A few years back I was interested in microsecond-precise timekeeping. My experience was GPS-backed network clocks were surprisingly cheap - but PTP-aware network switches were surprisingly expensive. And though I thought I could cut the cost of a grandmaster clock from $2000 down to $200, I figured there'd be no point if I then have to connect it to a $20,000 network switch. I wonder how Facebook are connecting their…
This gear will get cheaper soon.
There is a big push in the Industrial automation world to get TSN/PTP deployed more widely and perhaps universally. The idea is to eliminate the dichotomy of real time vs non real time Ethernet networks in control systems from the plant network down to the machine level.
This way you can send EtherCAT frames at equidistant intervals to servo drives in the microsecond range while casually streaming SPC data or Netflix over the same wire.
This is also in conjunction with the newer single pair Ethernet tech that the Automotive world is also interested in.
Re: Open-sourcing a more precise time appliance
#29> While building our Time Appliance, we also invented a Time Card, a PCIe card that can turn any commodity server into a time appliance. Invented? These cards have been around. e.g. Meinberg makes a variety of expansion cards which use various time keeping sources such as WWVB, IRIG-B, GNSS, etc.
Those have the external source but don't seem to have the internal reference (Rubidium, Cesium, etc) for holdover.
FB seems to be claiming to have invented a PCI card that makes a commodity server into a Stratum-1 source. Meinburg sells that too, but in proprietary appliance form.
Re: Open-sourcing a more precise time appliance
#30I'm still unclear why they need the MAC on top of the GNSS reception. I can understand if it's just to provide holdover in event of GNSS signal loss, but they seem to be saying that it provides better precision when you've got both.
That’s a good question.
GNSS gives you a 1 pulse per second signal. It is often with 20ns of accuracy. Now you need to interpolate nanoseconds between consecutive 1pps pulses. This is where a high stability oscillator is needed. You can certainly use an OCXO for the fraction of the cost of a MAC. We went for the MAC because of the importance of the role that the device plays in our datacenter