Cloudflare Time Services • Hacker News Board

Re: Cloudflare Time Services

#51

Post by wmf » Fri, Jun 21, 2019, 6:23 PM UTC

apeace wrote: post #27

jrockway wrote: post #4

Are the local time servers stratum 1, or do they have some central stratum 1 source that they then distribute out to the edge datacenters? What's the value over just building your own stratum 1 source? (Shameless plug: https://github.com/jrockway/beaglebone-gps-clock/blob/master... )

Many folks these days are deploying in the cloud, where it's difficult to deploy your own hardware. Many would see it as too costly to maintain the hardware in a separate datacenter with a fast connection to their cloud provider. The advantage of this is that it runs on Cloudflare's network, where your cloud provider surely has a fast connection to the nearest edge.

If you're in the cloud you should get time from the cloud provider, not from Cloudflare.

Re: Cloudflare Time Services

#52

Post by microcolonel » Fri, Jun 21, 2019, 6:24 PM UTC

xmichael999 wrote: post #42

redm wrote: post #2

It seems to me that CloudFlare should just sponsor servers for inclusion in the long-standing global NTP pools. https://www.ntppool.org/en/

CloudFlare is against anything open source. They seem like they are a nice sharing company but think again. They like Amazon are Open source thieves.

Dunno man, they release tremendous amounts of code, and pump huge resources into audits and other things which often go unaddressed in public projects. Aside from a moment of executive weakness a few years ago, they've been very principled and extremely generous.

I'm a bit concerned that they may become unprincipled and tyrannical once they are critical infrastructure, but that's a separate matter. Google is extremely generous as well, but as of late they are also disturbingly meddlesome.

Re: Cloudflare Time Services

#53

Post by Spivak » Fri, Jun 21, 2019, 6:24 PM UTC

xmichael999 wrote: post #42

redm wrote: post #2

It seems to me that CloudFlare should just sponsor servers for inclusion in the long-standing global NTP pools. https://www.ntppool.org/en/

CloudFlare is against anything open source. They seem like they are a nice sharing company but think again. They like Amazon are Open source thieves.

Would it even matter if they use OSS software to speak an open protocol for OSS clients?

It's not like your freedom is being restricted by their software licensing, if it was going to hurt anyone it would be CF themselves.

Stealing OSS software is kind of an oxymoron.

Re: Cloudflare Time Services

#54

Post by mentat » Fri, Jun 21, 2019, 6:27 PM UTC

redm wrote: post #2

It seems to me that CloudFlare should just sponsor servers for inclusion in the long-standing global NTP pools. https://www.ntppool.org/en/

Target accuracy for pool inclusion is not sufficient for many use cases. I've personally seen tens to 100s of ms difference between pool members. With no quality filter that's strict enough, adding more to that pool doesn't help.

Re: Cloudflare Time Services

#55

Post by jjoergensen » Fri, Jun 21, 2019, 6:59 PM UTC

dsp wrote: post #23

jjoergensen wrote: post #7

I assume Cloudflare does not serve leap-smeared time? In regards to rough-time, does anyone know what the result would be when both Cloudflare and Google are used at the same time, and Google serves leap-smeared time?

Leap smear is how true time is defined in the roughtime spec.

Thanks! I guess that answers my question. So the timing of both sources should match each other.

Re: Cloudflare Time Services

#56

Post by SEJeff » Fri, Jun 21, 2019, 7:17 PM UTC

mlichvar wrote: post #40

SEJeff wrote: post #32

Earlier quoted context omitted.

Yeah I agree, but ptpv2 does support unicast and you can set offsets. It would be very challenging, but not impossible sans the very high res stuff where you need oscillators on the switches and routers, as you alluded to. It wouldn't get down to the sub-10 nanosecond sync you get with a proper stratum 1 timesource (such as the rubidium decay ones), but you could get faster than the guaranteed 1 second of accuracy wh…

PTP over Internet doesn't make much sense. PTP requires hardware support in all network devices on the path between the (grand)master and slave. Without this support it will generally perform worse than NTP. Of course, it depends also on the implementation. PTP does support unicast messaging, but it is not meant to be used as a public service. There are two major problems: It's not stateless and it has a huge traffic…

Yeah this is the most sensible reason against it. NTP is also responsible for some of the biggest traffic amplification DDoS events.

Re: Cloudflare Time Services

#57

Post by akjj » Fri, Jun 21, 2019, 8:09 PM UTC

wbl wrote: post #37

Hello, I'm one of the engineers that worked on it. Feel free to ask me questions.

If I remember correctly, you're running roughenough [1] for the roughtime server, right? What software are you running for NTS?

[1] https://github.com/int08h/roughenough

Re: Cloudflare Time Services

#58

Post by mlichvar » Fri, Jun 21, 2019, 8:34 PM UTC

SEJeff wrote: post #56

mlichvar wrote: post #40

Earlier quoted context omitted.

PTP over Internet doesn't make much sense. PTP requires hardware support in all network devices on the path between the (grand)master and slave. Without this support it will generally perform worse than NTP. Of course, it depends also on the implementation. PTP does support unicast messaging, but it is not meant to be used as a public service. There are two major problems: It's not stateless and it has a huge traffic…

Yeah this is the most sensible reason against it. NTP is also responsible for some of the biggest traffic amplification DDoS events.

Yes, but NTP as a time service (client/server mode) is safe. A request has a single response and their lengths are symmetric (that's actually a requirement for accurate synchronization). The problem with amplification is in the optional monitoring/control modes of the protocol (modes 6 and 7 as used by the ntpq and ntpdc utilities respectively), which should be disabled on public servers. Unfortunately, there are still some old misconfigured servers causing problems for a lot of people.

In PTP the problem is in the synchronization protocol itself. A master in the unicast mode is basically a programmable packet generator. It sends sync/announce messages at a rate and duration specified by its slaves, and the address can be spoofed.

Re: Cloudflare Time Services

#59

Post by rkeene2 » Fri, Jun 21, 2019, 8:41 PM UTC

SEJeff wrote: post #13

I think it would be super interesting if they started offering IEEE 1588 v2 aka Precision Time Protocol, but it would be much harder to offer compared to ntp. What they're almost certainly doing here is just running a cable to the roof where they have a GPS antenna and then run it into their datacenters into a time appliance (less likely) or a GPS pci card in one of their servers (more likely) that they then send out…

For a project I was working I needed to synchronize clocks with extremely high precision (though accuracy was less important, and it was masterless so there was no authority) similar to PTP and I created the following simple system:

1. Server: Connect over TCP. For every byte sent, responds back with that system's time in microsceonds.

2. Client: Select a few peers, one at a time: connect over TCP, send a byte and note the round-trip time, halve it to get an approximation of the timepoint that the value was sent to you at, compute delta from current system clock; repeat 20 times, discard first 5 values; exclude outliers (+/- 1 standard deviation) and average to compute the offset, then repeat for the next host, etc until you get some change to make

It worked out and kept the required precision among the collection of nodes required for Ceph to work well.

Re: Cloudflare Time Services

#60

Post by makomk » Fri, Jun 21, 2019, 8:45 PM UTC

wmf wrote: post #51

apeace wrote: post #27

Earlier quoted context omitted.

Many folks these days are deploying in the cloud, where it's difficult to deploy your own hardware. Many would see it as too costly to maintain the hardware in a separate datacenter with a fast connection to their cloud provider. The advantage of this is that it runs on Cloudflare's network, where your cloud provider surely has a fast connection to the nearest edge.

If you're in the cloud you should get time from the cloud provider, not from Cloudflare.

That does seem like the obvious problem with this. For example AWS apparently have GPS time sync to all their datacenters in order to provide accurate NTP time to the instances in them: https://aws.amazon.com/blogs/aws/keeping-time-with-amazon-ti... It seems like it'd be hard to beat the connection speed and all-around performance of that, especially since there's a cross-internet trip from the third party stratum 1 servers to Cloudflare as well as the one from their servers to you.