Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Precision Time Protocol is being deployed at Meta

engineering.fb.com

Post reply on HN
Print view View original

81–90 of 104 posts

Re: How Precision Time Protocol is being deployed at Meta

#81

Post by lifeisstillgood »

klodolph wrote: post #15
throw0101a wrote: post #7

So they state: > One could argue that we don’t really need PTP for that. NTP will do just fine. Well, we thought that too. But experiments we ran comparing our state-of-the-art NTP implementation and an early version of PTP showed a roughly 100x performance difference: While I'm not necessarily against more accuracy/precision, what problems specifically are experiencing? They do mention some use cases of course: > Th…

I would also love to see an explanation of “why do we need this much accuracy?” that actually goes through the derivation of how much accuracy you need. Some of the justification for Google’s TrueTime is found in the Spanner docs: https://cloud.google.com/spanner/docs/true-time-external-con... Basically, you want to be able to do a “snapshot read” of the database rather than acquiring a lock (for reasons which should…

This is something like my third attempt to read the spanner paper - I get how it helps ordering of transactions but I am confused if it is used in making transactions atomic across machines ?

Re: How Precision Time Protocol is being deployed at Meta

#82

Post by hoseja »

akira2501 wrote: post #78
throw0101a wrote: post #7

So they state: > One could argue that we don’t really need PTP for that. NTP will do just fine. Well, we thought that too. But experiments we ran comparing our state-of-the-art NTP implementation and an early version of PTP showed a roughly 100x performance difference: While I'm not necessarily against more accuracy/precision, what problems specifically are experiencing? They do mention some use cases of course: > Th…

> But given that NTP (either ntpd or chrony) tends to give me an estimated error of around (tens of) 1e-6 seconds Is that a hard bound or an average? If it's an average, then what are the limits of the bounds, both in magnitude and duration? > and PTP can get down to 1e-9 seconds We use it for audio, and the reason it works well there is because there is no exponential backoff with your peers, allowing for even small…

I wouldn't call it absurdly simple.

Re: How Precision Time Protocol is being deployed at Meta

#83

Post by stefantalpalaru »

throw0101a wrote: post #7

So they state: > One could argue that we don’t really need PTP for that. NTP will do just fine. Well, we thought that too. But experiments we ran comparing our state-of-the-art NTP implementation and an early version of PTP showed a roughly 100x performance difference: While I'm not necessarily against more accuracy/precision, what problems specifically are experiencing? They do mention some use cases of course: > Th…

No post body was provided.

Re: How Precision Time Protocol is being deployed at Meta

#84

Post by throw0101a »

lifeisstillgood wrote: post #81
klodolph wrote: post #15

Earlier quoted context omitted.

I would also love to see an explanation of “why do we need this much accuracy?” that actually goes through the derivation of how much accuracy you need. Some of the justification for Google’s TrueTime is found in the Spanner docs: https://cloud.google.com/spanner/docs/true-time-external-con... Basically, you want to be able to do a “snapshot read” of the database rather than acquiring a lock (for reasons which should…

This is something like my third attempt to read the spanner paper - I get how it helps ordering of transactions but I am confused if it is used in making transactions atomic across machines ?

> I get how it helps ordering of transactions but I am confused if it is used in making transactions atomic across machines ?

AIUI, you cannot quiet think of it like a regular database where a particular row has a particular value which would necessitate only one writer doing (atomic) updates at a time.

Rather it is a MVCC-like database and a bit like an append-only log: as many writers as needed can write and there are 'multiple values' for each row. The "actual" value of the row is the one with the highest transaction ID / timestamp. So updates can happening without (atomic) locking by just adding to the value(s) that already exist.

When reading, applications just generally get served the value with the highest-value timestamp, and since time is synchronized to such a tiny interval, it is a reasonably sure bet that the highest value is the most recent transaction.

This is similar in concept to a vector clock (see also Lamport):

* https://en.wikipedia.org/wiki/Vector_clock

But instead of logical clocks with 'imaginary time', 'real time' is used down to the sub-microsecond level.

Re: How Precision Time Protocol is being deployed at Meta

#85

Post by throw0101a »

akira2501 wrote: post #78
throw0101a wrote: post #7

So they state: > One could argue that we don’t really need PTP for that. NTP will do just fine. Well, we thought that too. But experiments we ran comparing our state-of-the-art NTP implementation and an early version of PTP showed a roughly 100x performance difference: While I'm not necessarily against more accuracy/precision, what problems specifically are experiencing? They do mention some use cases of course: > Th…

> But given that NTP (either ntpd or chrony) tends to give me an estimated error of around (tens of) 1e-6 seconds Is that a hard bound or an average? If it's an average, then what are the limits of the bounds, both in magnitude and duration? > and PTP can get down to 1e-9 seconds We use it for audio, and the reason it works well there is because there is no exponential backoff with your peers, allowing for even small…

> Another advantage of PTP is it can operate as a broadcast

NTP can work via broadcast and multicast.

Re: How Precision Time Protocol is being deployed at Meta

#86

Post by throw0101a »

stingraycharles wrote: post #80
throw0101a wrote: post #7

So they state: > One could argue that we don’t really need PTP for that. NTP will do just fine. Well, we thought that too. But experiments we ran comparing our state-of-the-art NTP implementation and an early version of PTP showed a roughly 100x performance difference: While I'm not necessarily against more accuracy/precision, what problems specifically are experiencing? They do mention some use cases of course: > Th…

> But given that NTP (either ntpd or chrony) tends to give me an estimated error of around (tens of) 1e-6 seconds, and PTP can get down to 1e-9 seconds, I'm not sure how many data centre applications need that level of accuracy. I know that in trading, auditing trades / order books requires extremely accurate timing, and they typically deploy GPS hardware to get the required level of accuracy. As GPS is accurate to a…

> […] improvement needed to not need GPS hardware anymore.

You're simply trading NTP hardware for PTP hardware (grandmaster clocks). There is no way to get to 1e-9 scales without hardware support.

Re: How Precision Time Protocol is being deployed at Meta

#87

Post by jimmySixDOF »

foobiekr wrote: post #18
CapmCrackaWaka wrote: post #6

Why do so many tech companies seem to be releasing “secret sauce” for free lately? I see a lot of posts lately detailing how inner production systems work at large companies, and while I’m grateful, I’m curious why the higher ups think it’s worthwhile to release this information.

PTP isn’t secret sauce. Routers I worked on were doing PTP in like 2009. 1588 was standardized in 2002.

There was the TICTOC workgroup, PWE3 for psuedowires on MPLS and whole labyrinth of IEC standards not to mention closed ring optical network clock syncs from SONNET/SDH. I suppose its secret sauce for social media software.

Re: How Precision Time Protocol is being deployed at Meta

#88

Post by elcomet »

capableweb wrote: post #51
elcomet wrote: post #47

Earlier quoted context omitted.

You could apply to work on the research infrastructure (for instance for AI)

And instead of writing infrastructure for the developers who write code for ads, you now write AI software/do AI research for a company in order to attract enough talent so they can hire developers to write infrastructure for the developers who write code for ads? Or you know, just don't work for companies who tend to collect as much user data as they can?

But at least you're contributing to science

Re: How Precision Time Protocol is being deployed at Meta

#89

Post by matheweis »

vanshg wrote: post #55

> It’s absolutely essential to ensure an open sky and install a solid stationary antenna What effect, if any, might a major earthquake have on the time calculation?

That’s a very interesting question.

1ns of error equates to ~30cm of error (from speed of light), so:

You might get a couple of ns of error from the initial s and p waves. (Assuming on the order of 50cm displacement)

If you’re on the fault and there is a major displacement (say order of 1-2 meters) there might be several more ns of error.

Practically, this seems to suggest that by the time that displacement from an earthquake would cause a significant loss of clock accuracy (say > 10ns), you most likely have bigger problems to worry about..

Post reply on HN