Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Clock synchronization is a nightmare

arpitbhayani.me

Post reply on HN
Print view View original

71–80 of 165 posts

Re: Clock synchronization is a nightmare

#71

Post by raron »

jaggederest wrote: post #60
amelius wrote: post #57

Earlier quoted context omitted.

Same here. I wish there was an easy way around it (that doesn't require me to play sysadmin in my spare time).

In the 80s my uncle had digital clocks that used an antenna to tune into the atomic clock time signal that (was/is?) broadcast nationwide. I've long wished that it was incorporated into stoves, microwaves, essentially everything that isn't an internet device (yet... sigh) Sadly I think the actual antenna and hardware were relatively large since it's a long wave signal, but maybe with SDR it'll all fit on the head of…

> atomic clock time signal that (was/is?) broadcast nationwide

Probably DCF77 or WWVB.

> I think the actual antenna and hardware were relatively large since it's a long wave signal

Casio has some normal sized wristwatches that synchronizes to DCF77, it would definitely fit into a stove, microwave, or basically anything.

Re: Clock synchronization is a nightmare

#72

Post by Enginerrrd »

nuccy wrote: post #64
shomp wrote: post #50

Absolute synchronization impossible?? Challenge accepted.

Nature (laws of physics) is agains you on this: it is in fact impossible for everyone. What is in sync for some observers can be out of sync for others (depends on where they are, i.e. gravity, and how they relatively move). See general and special relativity principle of simultaneity [1]. 1. https://en.wikipedia.org/wiki/Relativity_of_simultaneity

I think you just nerd-sniped me but I’m not convinced it’s impossible to assign a consistent ordering to events with relativistic separations.

For starters, the spacetime interval between two events IS a Lorentz invariant quantity. That could probably be used to establish a universal order for timelike separations between events. I suspect that you could use a reference clock, like a pulsar or something to act as an event against which to measure the spacetime interval to other events, and use that for ordering. Any events separated by a light-like interval are essentially simultaneous to all observers under that measure.

The problem comes for events with a space like or light like separation. In that case, the spacetime interval is still conserved, but I’m not sure how you assign order to them. Perhaps the same system works without modification, but I’m not sure.

Re: Clock synchronization is a nightmare

#73

Post by dmazin »

I highly recommend anyone to look up how PTP works and how it compares to NTP. Clock sync is very interesting. When I joined an HFT company, first thing I did was understand this stuff. We care about it a lot[1].

If you want a specific question to answer, answer this: why does PTP need hardware timestamping to achieve high precision (where the network card itself assigns timestamps to packets, rather than having the kernel do it as part of TCP/IP processing)? If we use software timestamps, why can we do microsecond precision at best? If you understand this, it goes a very long way to understanding the core ideas behind precise clock sync.

Once you have a solid understanding of PTP, look into White Rabbit. They’re able to sync two clocks with sub-ns precision. In case that isn’t obvious, that is absolutely insane.

[1] So do a lot of people. For example audio engineers. Once, an audio engineer absolutely talked my ear off about ptp. I had no idea that audio people understood clock sync so well but they do!

Re: Clock synchronization is a nightmare

#74

Post by [deleted] »

jasonwatkinspdx wrote: post #51
NelsonMinar wrote: post #48

On the flipside, clock sync for civilians has never been easier. Thanks to NTP any device with an Internet connection can pretty easily get time accurate to 1 second, often as little as 10 ms. All major consumer computers are preconfigured to sync time to one of several reliable NTP pools. This post is about more complicated synchronization for more demanding applications. And it's very good. I'm just marveling at ho…

At this point the only clock in my life that doesn't auto set is the one on my stove, and that's because I abhor internet connected kitchen appliances.

[deleted]

Re: Clock synchronization is a nightmare

#75

Post by throw0101c »

structural wrote: post #59
idorosen wrote: post #27

Earlier quoted context omitted.

Alternatively, you could guarantee the same synchronization using PPS and PTP to each host's DCD pin of their serial port or to specialized hardware such as modern PTP-enabled smart NICs/FPGAs that can accept PPS input. GPS+PPS gets you to within 20-80ns global synchronization depending on implementation (assuming you're all mostly in the same inertial frame), and allows you to make much stronger guarantees than True…

Of course, you can do this in good conditions. The extremely powerful part that TrueTime brings is how the system degrades when something goes wrong. If everyone is synced to +/- 20ns, that's great. Then when someone flies over your datacenter with an GPS jammer (purposeful or accidental), this needs to not be a bad day where suddenly database transactions happen out of order, or you have an outage. The other benefit…

> Then when someone flies over your datacenter with an GPS jammer (purposeful or accidental), this needs to not be a bad day where suddenly database transactions happen out of order, or you have an outage.

Most NTP/PTP appliances have internal clocks that are OCXO or rubidium that have holdover (even for several days).

If time is that important to you then you'll have them, plus perhaps some fibre connections to other sites that are hopefully out of range of the jamming.

Re: Clock synchronization is a nightmare

#76

Post by lll-o-lll »

josephg wrote: post #55
lll-o-lll wrote: post #31

Earlier quoted context omitted.

Yes. A consistent total ordering is what you need (want) in distributed computing. Ultimately, causality is what is important, but consistent ordering of concurrent operations makes things much easier to work with.

Consistent ordering of concurrent operations is easy though. Just detect this case (via logical clocks) then order using node ids or transaction ids if the logical clocks show the transactions as being concurrent. Am I missing something? This feels like a very solved problem. (I’ve worked on CRDTs where we have the same problem. There exist incredibly fast algorithms for this.)

> Am I missing something?

I don’t think so, I think it is solved in the general sense. However what Spanner does is unique, and it does use synchronised clocks in order to do it.

However, Spanner does not solve the inter-continental acid database with high write throughput. So I don’t see it as ground breaking. CRDT’s are interesting, I’ve followed your work for a long time, but too constrained to solve this general problem I think.

Re: Clock synchronization is a nightmare

#77

Post by Uehreka »

NelsonMinar wrote: post #48

On the flipside, clock sync for civilians has never been easier. Thanks to NTP any device with an Internet connection can pretty easily get time accurate to 1 second, often as little as 10 ms. All major consumer computers are preconfigured to sync time to one of several reliable NTP pools. This post is about more complicated synchronization for more demanding applications. And it's very good. I'm just marveling at ho…

I was doing something at work that involved calculating round trip times from/to Android devices, and learned that although it should be possible for NTP to sync clocks with below-second precision, in practice many of the Android devices I was working with (mostly Pixels 2-7) were off from my server and each other by up to 5 seconds, which blew my mind.

Re: Clock synchronization is a nightmare

#78

Post by jaggederest »

raron wrote: post #71
jaggederest wrote: post #60

Earlier quoted context omitted.

In the 80s my uncle had digital clocks that used an antenna to tune into the atomic clock time signal that (was/is?) broadcast nationwide. I've long wished that it was incorporated into stoves, microwaves, essentially everything that isn't an internet device (yet... sigh) Sadly I think the actual antenna and hardware were relatively large since it's a long wave signal, but maybe with SDR it'll all fit on the head of…

> atomic clock time signal that (was/is?) broadcast nationwide Probably DCF77 or WWVB. > I think the actual antenna and hardware were relatively large since it's a long wave signal Casio has some normal sized wristwatches that synchronizes to DCF77, it would definitely fit into a stove, microwave, or basically anything.

I believe it was a longwave broadcast so probably WWVB which would apparently imply a 60mm antenna, but it was a standard old school "GE digital clock radio" form factor so size wasn't at a premium.

Re: Clock synchronization is a nightmare

#79

Post by georgelyon »

josephg wrote: post #55
lll-o-lll wrote: post #31

Earlier quoted context omitted.

Yes. A consistent total ordering is what you need (want) in distributed computing. Ultimately, causality is what is important, but consistent ordering of concurrent operations makes things much easier to work with.

Consistent ordering of concurrent operations is easy though. Just detect this case (via logical clocks) then order using node ids or transaction ids if the logical clocks show the transactions as being concurrent. Am I missing something? This feels like a very solved problem. (I’ve worked on CRDTs where we have the same problem. There exist incredibly fast algorithms for this.)

Yes, though the API of having a write-only value that is a monotonically increasing counter is much simpler than having to think about causality or logical clocks.
Post reply on HN