The school for sysadmins who can’t timesync good
libertysys.com.au
The school for sysadmins who can’t timesync good
1–10 of 33 posts
Re: The school for sysadmins who can’t timesync good
#2Re: The school for sysadmins who can’t timesync good
#3Nice comprehensive series but couldn't get to page 4 -site timed out. On the windows side of things, I'm more familiar with "w32tm" and "net time." My time sync post has the highest amount of views on my site from people googling "how to set time clock on domain" so their cell phones match their computers at work. Would be interesting to see how the windows protocols differ from nix.
Re: The school for sysadmins who can’t timesync good
#4Nice comprehensive series but couldn't get to page 4 -site timed out. On the windows side of things, I'm more familiar with "w32tm" and "net time." My time sync post has the highest amount of views on my site from people googling "how to set time clock on domain" so their cell phones match their computers at work. Would be interesting to see how the windows protocols differ from nix.
Of course on Linux most of the arcane details of the ntp daemon aren't relevant because most distros end up running SystemD with timesyncd instead. I discovered this when all of my T1 time sources (GPS receivers) stopped working after an update. As usual you can disable the systemd bit, but it doesn't like it.
Re: The school for sysadmins who can’t timesync good
#5Nice comprehensive series but couldn't get to page 4 -site timed out. On the windows side of things, I'm more familiar with "w32tm" and "net time." My time sync post has the highest amount of views on my site from people googling "how to set time clock on domain" so their cell phones match their computers at work. Would be interesting to see how the windows protocols differ from nix.
The Win32 daemon only provides coarse time adjustment. Basically doing an ntpdate to adjust the clocks once per day or so. Good enough for domain logins, but a couple orders of magnitude worse than the regular NTP protocol. Of course on Linux most of the arcane details of the ntp daemon aren't relevant because most distros end up running SystemD with timesyncd instead. I discovered this when all of my T1 time sources…
which tends to be "good enough" for the average desktop user, but anything even vaguely server-ish should run a full NTP implementation such as chrony and not an SNTP one such as systemd-timesyncd.
a few years ago at $dayjob we had a fleet of CoreOS hosts. CoreOS, at the time, defaulted to systemd-timesyncd using pool.ntp.org addresses.
our CoreOS hosts, obviously, ran Docker containers.
systemd has a neat "feature" where if your network configuration changes, it'll trigger a time synchronization through timesyncd.
when a new Docker container was started, this counted as a "network config change" and caused a time synchronization.
by itself, this isn't too bad. it caused time syncs to happen more often than they need to, strictly speaking, but shouldn't have caused any further problems.
except...enter "falsetickers". hosts in the NTP pool are run by volunteers. an individual host in the pool may have the incorrect time.
the infrastructure for the NTP pool has monitoring for this, and will kick a host out of the DNS rotation if it's wrong. except this won't happen immediately - there'll always be some lag between when the host starts being wrong and when the monitoring system kicks it out.
and if your hosts are synchronizing their time more often than necessary, it increases the chance they'll do a time sync in one of these small windows where a falseticker is being advertised by the pool.
a full NTP implementation is specifically designed to handle this, of course. a client polls multiple servers, and will discard significant outliers.
SNTP? not so much. I haven't looked at timesyncd to see if it's improved since then, but at the time it would pick one of the [0-3].pool.ntp.org hosts at random, send it one NTP packet, and then jump the time to that response.
...and that's the story of how some of my company's production hosts would have their system time autonomously jump to be 5-10 minutes fast, maintain that time for several minutes to an hour, and then jump back to the correct time, all without human intervention.
Re: The school for sysadmins who can’t timesync good
#6Re: The school for sysadmins who can’t timesync good
#7Hrm, couldn't get to page 3. Guess he needs a school for scaling static content delivery.
Re: The school for sysadmins who can’t timesync good
#8Earlier quoted context omitted.
The Win32 daemon only provides coarse time adjustment. Basically doing an ntpdate to adjust the clocks once per day or so. Good enough for domain logins, but a couple orders of magnitude worse than the regular NTP protocol. Of course on Linux most of the arcane details of the ntp daemon aren't relevant because most distros end up running SystemD with timesyncd instead. I discovered this when all of my T1 time sources…
> most distros end up running SystemD with timesyncd instead which tends to be "good enough" for the average desktop user, but anything even vaguely server-ish should run a full NTP implementation such as chrony and not an SNTP one such as systemd-timesyncd. a few years ago at $dayjob we had a fleet of CoreOS hosts. CoreOS, at the time, defaulted to systemd-timesyncd using pool.ntp.org addresses. our CoreOS hosts, ob…
Then at least if I’m off we’re all off together.
Re: The school for sysadmins who can’t timesync good
#9Earlier quoted context omitted.
The Win32 daemon only provides coarse time adjustment. Basically doing an ntpdate to adjust the clocks once per day or so. Good enough for domain logins, but a couple orders of magnitude worse than the regular NTP protocol. Of course on Linux most of the arcane details of the ntp daemon aren't relevant because most distros end up running SystemD with timesyncd instead. I discovered this when all of my T1 time sources…
> most distros end up running SystemD with timesyncd instead which tends to be "good enough" for the average desktop user, but anything even vaguely server-ish should run a full NTP implementation such as chrony and not an SNTP one such as systemd-timesyncd. a few years ago at $dayjob we had a fleet of CoreOS hosts. CoreOS, at the time, defaulted to systemd-timesyncd using pool.ntp.org addresses. our CoreOS hosts, ob…
Re: The school for sysadmins who can’t timesync good
#10Earlier quoted context omitted.
> most distros end up running SystemD with timesyncd instead which tends to be "good enough" for the average desktop user, but anything even vaguely server-ish should run a full NTP implementation such as chrony and not an SNTP one such as systemd-timesyncd. a few years ago at $dayjob we had a fleet of CoreOS hosts. CoreOS, at the time, defaulted to systemd-timesyncd using pool.ntp.org addresses. our CoreOS hosts, ob…
I cheat and have an authoritative NTP server locally and then override dns for pool.ntp.org and friends. Then at least if I’m off we’re all off together.