Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FPGA NTP Server

research.wand.net.nz

Post reply on HN
Print view View original

31–40 of 70 posts

Re: FPGA NTP Server

#31

Post by varjag »

mlichvar wrote: post #25
hummo56 wrote: post #22

Interesting project, especially to gain insights into the FPGA programming part. I think it is essentially a mixture of PTP and NTP now. I guess this will work within the same local network, as the major inaccuracy of NTP comes from the asymmetric path delays at the network layer over the Internet. PTP solves this by incorporating these hardware timestamps exactly. But this works only within the same LAN. Here is an…

The main difference between PTP and NTP is that PTP relies on hardware support in switches and routers. Those are not cheap. If they had the same support for NTP, it would perform as well. A highly accurate stratum-1 NTP server can be build with a common computer NIC. No need to mess with FPGAs (unless that's your thing). The Intel I210 is about $50. It has a PPS input and output. With some calibration, the timestamp…

PTP at this point is ubiquitous and cheap. It is hard to source a PHY that has no support for it in 2021.

> If they had the same support for NTP, it would perform as well.

Unless you'd extend NTP to become PTP, it absolutely wouldn't.

Re: FPGA NTP Server

#32

Post by HourglassFR »

markfeathers wrote: post #14
wmf wrote: post #13

Earlier quoted context omitted.

To do PTP "right" requires every switch to support it and a NIC with hardware timestamps. Also, I've seen claims that PTP is no more precise than a good implementation of NTP.

ptp is If you look at the white rabbit FPGA PTP updates, its in the ns range. Any kind of GPS + most intel nics will get you PTP with an accurate clock. If you didn't need to sync too many devices you could use a single system with a bunch of nics as your "switch".

> If you look at the white rabbit FPGA PTP updates, its in the ns range

As I recall, I had even better performance than that. Around the tens of picoseconds. But I guess the advertised 1 ns is a conservative estimate. The precison is incredible but its not magic, they squeeze the maximum amount of determinism out of custom hardware and fiber optic links. It is a bit of pain too set up, as you need to calibrate each link individually every time you change the fiber or the SFP.

Re: FPGA NTP Server

#33

Post by gbrown_ »

lmilcin wrote: post #27

> Microprocessor based Network Time Protocol (NTP) servers suffer from a large amount of timestamp jitter, due to the hardware and Operating System (OS) being shared among other applications. So instead of configuring Linux correctly and dedicating a core for your NTP you have created custom hardware. Congratulations, now you have two problems. I have been working on algorithmic trading and it is not that hard to rel…

> dedicating a core for your NTP

Heh as soon as I read this I thought "algo trader?". If you don't mind me asking outside of trading how often do you see systems seriously manage jitter (specifically I'm thinking about interrupt management). I'm in HPC and whilst some such management is present it seems seldom thought about.

Re: FPGA NTP Server

#35

Post by lmilcin »

gbrown_ wrote: post #33
lmilcin wrote: post #27

> Microprocessor based Network Time Protocol (NTP) servers suffer from a large amount of timestamp jitter, due to the hardware and Operating System (OS) being shared among other applications. So instead of configuring Linux correctly and dedicating a core for your NTP you have created custom hardware. Congratulations, now you have two problems. I have been working on algorithmic trading and it is not that hard to rel…

> dedicating a core for your NTP Heh as soon as I read this I thought "algo trader?". If you don't mind me asking outside of trading how often do you see systems seriously manage jitter (specifically I'm thinking about interrupt management). I'm in HPC and whilst some such management is present it seems seldom thought about.

Heh, I already thought somebody will point it out.

See, they decided to create custom hardware just to do NTP. How is instead dedicating a single core a worse solution? Setting dedicated core is basically configuration detail.

There are no gains from custom hardware as network variability would mask them. If you want really good time guarantee because having right time is critical for your application, you put atomic clocks in your servers as one well known company does.

Also, it is customary to have dedicated machines for various functions. You would not normally want to mix some different types of loads that conflict with each other, for example high security with low security, high throughput with real time, etc.

You could just configure a single machine dedicated to NTP (which is customary), but instead of sharing cores between OS and NTP (and introducing jitter) you can have separate core for the OS and separate for NTP so that NTP can work undisturbed.

So if you maintain an NTP instance and have problem with jitter and you red this article my hint is: don't set up custom FPGA to fix your problem, just dedicate single core on a machine you already have and you will get as good results as you can.

Re: FPGA NTP Server

#36

Post by hansor »

gbrown_ wrote: post #33
lmilcin wrote: post #27

> Microprocessor based Network Time Protocol (NTP) servers suffer from a large amount of timestamp jitter, due to the hardware and Operating System (OS) being shared among other applications. So instead of configuring Linux correctly and dedicating a core for your NTP you have created custom hardware. Congratulations, now you have two problems. I have been working on algorithmic trading and it is not that hard to rel…

> dedicating a core for your NTP Heh as soon as I read this I thought "algo trader?". If you don't mind me asking outside of trading how often do you see systems seriously manage jitter (specifically I'm thinking about interrupt management). I'm in HPC and whilst some such management is present it seems seldom thought about.

NTP is hilariously insufficient for HFT, we have far better protocol around: PTP.

https://en.wikipedia.org/wiki/Precision_Time_Protocol

Re: FPGA NTP Server

#37

Post by mlichvar »

varjag wrote: post #31
mlichvar wrote: post #25

Earlier quoted context omitted.

The main difference between PTP and NTP is that PTP relies on hardware support in switches and routers. Those are not cheap. If they had the same support for NTP, it would perform as well. A highly accurate stratum-1 NTP server can be build with a common computer NIC. No need to mess with FPGAs (unless that's your thing). The Intel I210 is about $50. It has a PPS input and output. With some calibration, the timestamp…

PTP at this point is ubiquitous and cheap. It is hard to source a PHY that has no support for it in 2021. > If they had the same support for NTP, it would perform as well. Unless you'd extend NTP to become PTP, it absolutely wouldn't.

Yes, NICs with support for hardware timestamping are common (it's typically in the MAC, not PHY), but switches that have a good support for PTP, either as a boundary clock, or transparent clock, are not cheap. At least I have not seen one yet. Do you have any examples?

Some switches support NTP as a server and client (equivalent to the PTP boundary clock), but there don't seem to be any using hardware timestamping. It's just the classic ntpd using software timestamps, good to few tens of microseconds at best.

And yes, NTP could definitely perform as well as PTP if the switches had a proper support. In my tests with directly connected NICs the synchronization is stable to few nanoseconds, same as with PTP. At the protocol level, they use the same timestamps.

Re: FPGA NTP Server

#38

Post by gbrown_ »

lmilcin wrote: post #35
gbrown_ wrote: post #33

Earlier quoted context omitted.

> dedicating a core for your NTP Heh as soon as I read this I thought "algo trader?". If you don't mind me asking outside of trading how often do you see systems seriously manage jitter (specifically I'm thinking about interrupt management). I'm in HPC and whilst some such management is present it seems seldom thought about.

Heh, I already thought somebody will point it out. See, they decided to create custom hardware just to do NTP. How is instead dedicating a single core a worse solution? Setting dedicated core is basically configuration detail. There are no gains from custom hardware as network variability would mask them. If you want really good time guarantee because having right time is critical for your application, you put atomic…

Oh I wasn't meaning specifically in the context of NTP, just generally.

Re: FPGA NTP Server

#39

Post by gbrown_ »

hansor wrote: post #36
gbrown_ wrote: post #33

Earlier quoted context omitted.

> dedicating a core for your NTP Heh as soon as I read this I thought "algo trader?". If you don't mind me asking outside of trading how often do you see systems seriously manage jitter (specifically I'm thinking about interrupt management). I'm in HPC and whilst some such management is present it seems seldom thought about.

NTP is hilariously insufficient for HFT, we have far better protocol around: PTP. https://en.wikipedia.org/wiki/Precision_Time_Protocol

I'm aware, I was talking about jitter in general.

Re: FPGA NTP Server

#40

Post by lmilcin »

gbrown_ wrote: post #38
lmilcin wrote: post #35

Earlier quoted context omitted.

Heh, I already thought somebody will point it out. See, they decided to create custom hardware just to do NTP. How is instead dedicating a single core a worse solution? Setting dedicated core is basically configuration detail. There are no gains from custom hardware as network variability would mask them. If you want really good time guarantee because having right time is critical for your application, you put atomic…

Oh I wasn't meaning specifically in the context of NTP, just generally.

> Oh I wasn't meaning specifically in the context of NTP, just generally.

So you decided to just answer with a generalization without regard for the particular case of NTP?

Are you frequently building custom hardware when default OS settings do not suit your needs?

Linux has a lot of configuration potential. It is wise to explore and understand various knobs available before you decide to complicate your life and build something very complex that could easily be replaced with a one line shell script.

Post reply on HN