Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FPGA NTP Server

research.wand.net.nz

Post reply on HN
Print view View original

61–70 of 70 posts

Re: FPGA NTP Server

#61

Post by mlyle »

lmilcin wrote: post #49
Unklejoe wrote: post #44

Earlier quoted context omitted.

> I have been working on algorithmic trading and it is not that hard to reliably (like 100% of the time) respond within couple microseconds. As someone who works in the flight test instrumentation industry (that primarily uses PTP rather than NTP), my first thought was "is a couple of microseconds supposed to be good?". With PTP, I usually achieve RMS offsets of I sort of agree with your point in that it's probably p…

On a "normal" box the time is usually combination of various sources neither of which was really designed to provide accurate, sub microsecond, absolute time. (That except possibly for this "PTP hardware" in NICs that I don't know anything about) So even if you somehow devise a piece of hardware to pass the accurate time to the operating system, there would be no way to track it accurately. Now, I think that really a…

Most Intel 10 gigabit adapters support PTP.

It's just about inserting the exact time the frame will be sent into a sent frame, and noting the exact time the frame is received. All that's left is a small amount of clock jitter and cable delays.

High end switches support PTP themselves so you don't need to worry about queueing delays, but you don't need a switch that supports PTP to get high resolution, low jitter, low uncertainty time-- at least on reasonably sized networks that are usually not saturated.

There are many algorithmic trading applications that rely upon high quality time derived from a single coherent source, along with telecommunications, control systems, instrumentation, etc. And there are other distributed systems approaches where true ordering is nice: yes, you can provide ordering within a system of dependent events using a Lamport clock, vector clock, etc, but without high quality time you can also correctly sequence causally related events originating outside your tightly coupled system in many use cases.

That said: PTP relies on a tightly coupled master and realistically is intended for a tightly coupled, hierarchal system. NTP is a better "internet" time protocol-- lots of logic for clock precedence, averaging of multiple sources, slower control system, etc.

Re: FPGA NTP Server

#62

Post by 1_person »

Unklejoe wrote: post #59
1_person wrote: post #55

Earlier quoted context omitted.

The precision achievable with FPGAs is well beyond 10s of nanoseconds. The high rate transceivers are used with capacitative dividers or internal signal propagation delays defining the reference intervals for time to digital conversion in extreme accuracy applications, these and similar approaches yield precision to picosecond scale and beyond.

Right, but Ethernet timestamping usually occurs on a symbol or byte level inside the MAC, so for GigE, you're looking at 8 nanoseconds per symbol. This could be improved by oversampling the symbols, but at least with GigE, there's (I think) 4 cycles of uncertainty each time the link comes up due to how the clocking agreement between master/slave is established. It's also somewhat difficult to run large counters faste…

The approaches I am referring to are indeed oversampled and occur in a "bump in the wire" or on a parallel data path.

Moreover, the oversampling occurs at a much higher frequency than the maximum counter frequency of the FPGA -- each clock period is divided into a fractional offset from the rising edge by the capacitative divider or delay line.

It is not uncommon at all for dedicated PTP/NTP hardware to implement synchronization and timestamping in this way, even commodity NICs supporting hardware timestamping often implement it on at least a parallel data path.

Re: FPGA NTP Server

#63

Post by willis936 »

sgtnoodle wrote: post #60
willis936 wrote: post #43

Earlier quoted context omitted.

Thank you for the detailed response! The PPS is currently driving a hardware interrupt on the raspberry pi that is read in by kernel mode software. My project is to drive an external display. Normally I would bypass the raspberry pi altogether and connect the PPS signal to the strobe input of the SIPO shift register. The problem is that the PPS signal cannot be trusted to always exist. Using a raspberry pi has a few…

If you want to precisely generate a PPS edge in software with less jitter than you can schedule, you can use a PWM peripheral. Wake up a few milliseconds before the PPS edge is due, get the system time, and compute the precise time until the PPS is due. Initialize the PWM peripheral to transition that far into the future, then go back to sleep until a bit after the transition should have happened, and disable the PWM…

Nixie tubes driven by a pair of cascaded HV5122 (driver + shift register). The strobe input is what updates the output registers with the recently shifted in contents. The driver takes 500 ns to turn on and the nixie tubes take about 10 us to fire once the voltage is applied.

I know it's absurd to worry about the last few ms, but it's part of what interests me about the project. The goal is to make The Wall Time as accurate as I can. I could go further with a delay locked loop fed from measuring nixie tube current. There is room push down to the dozens of nanoseconds of error relative to the PPS source, but I am content with the 10s of microseconds. I can't imagine ever having access to a camera that could capture that amount of error.

Thanks for the tip. Hardware timers are best. I'll likely have to take some measurements to calibrate the computation time of getting the system time and performing the subtraction.

Re: FPGA NTP Server

#64

Post by eqvinox »

mlyle wrote: post #61
lmilcin wrote: post #49

Earlier quoted context omitted.

On a "normal" box the time is usually combination of various sources neither of which was really designed to provide accurate, sub microsecond, absolute time. (That except possibly for this "PTP hardware" in NICs that I don't know anything about) So even if you somehow devise a piece of hardware to pass the accurate time to the operating system, there would be no way to track it accurately. Now, I think that really a…

Most Intel 10 gigabit adapters support PTP. It's just about inserting the exact time the frame will be sent into a sent frame, and noting the exact time the frame is received. All that's left is a small amount of clock jitter and cable delays. High end switches support PTP themselves so you don't need to worry about queueing delays, but you don't need a switch that supports PTP to get high resolution, low jitter, low…

A lot of intel 1G adapters do too, likely including the one in your laptop.

  $ lspci -s 0:1f.6
  00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (4) I219-V (rev 21)
  $ sudo ethtool -T enp0s31f6 
  Time stamping parameters for enp0s31f6:
  Capabilities:
   hardware-transmit
   software-transmit
   hardware-receive
   software-receive
   software-system-clock
   hardware-raw-clock
  PTP Hardware Clock: 0
  Hardware Transmit Timestamp Modes:
   off
   on
  Hardware Receive Filter Modes:
   none
   all
   ptpv1-l4-sync
   ptpv1-l4-delay-req
   ptpv2-l4-sync
   ptpv2-l4-delay-req
   ptpv2-l2-sync
   ptpv2-l2-delay-req
   ptpv2-event
   ptpv2-sync
   ptpv2-delay-req

Re: FPGA NTP Server

#65

Post by mlyle »

eqvinox wrote: post #64
mlyle wrote: post #61

Earlier quoted context omitted.

Most Intel 10 gigabit adapters support PTP. It's just about inserting the exact time the frame will be sent into a sent frame, and noting the exact time the frame is received. All that's left is a small amount of clock jitter and cable delays. High end switches support PTP themselves so you don't need to worry about queueing delays, but you don't need a switch that supports PTP to get high resolution, low jitter, low…

A lot of intel 1G adapters do too, likely including the one in your laptop. $ lspci -s 0:1f.6 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (4) I219-V (rev 21) $ sudo ethtool -T enp0s31f6 Time stamping parameters for enp0s31f6: Capabilities: hardware-transmit software-transmit hardware-receive software-receive software-system-clock hardware-raw-clock PTP Hardware Clock: 0 Hardware Transmit Timest…

Yup-- I didn't know the prevalence so I didn't say.

You're wrong though-- there's no ethernet adapter in my laptop ;)

Re: FPGA NTP Server

#66

Post by eqvinox »

mlyle wrote: post #65
eqvinox wrote: post #64

Earlier quoted context omitted.

A lot of intel 1G adapters do too, likely including the one in your laptop. $ lspci -s 0:1f.6 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (4) I219-V (rev 21) $ sudo ethtool -T enp0s31f6 Time stamping parameters for enp0s31f6: Capabilities: hardware-transmit software-transmit hardware-receive software-receive software-system-clock hardware-raw-clock PTP Hardware Clock: 0 Hardware Transmit Timest…

Yup-- I didn't know the prevalence so I didn't say. You're wrong though-- there's no ethernet adapter in my laptop ;)

Touché :D

(sadly, 802.11 timestamping on wifi is very fringe...)

Re: FPGA NTP Server

#67

Post by sgtnoodle »

willis936 wrote: post #63
sgtnoodle wrote: post #60

Earlier quoted context omitted.

If you want to precisely generate a PPS edge in software with less jitter than you can schedule, you can use a PWM peripheral. Wake up a few milliseconds before the PPS edge is due, get the system time, and compute the precise time until the PPS is due. Initialize the PWM peripheral to transition that far into the future, then go back to sleep until a bit after the transition should have happened, and disable the PWM…

Nixie tubes driven by a pair of cascaded HV5122 (driver + shift register). The strobe input is what updates the output registers with the recently shifted in contents. The driver takes 500 ns to turn on and the nixie tubes take about 10 us to fire once the voltage is applied. I know it's absurd to worry about the last few ms, but it's part of what interests me about the project. The goal is to make The Wall Time as a…

Sounds like fun! For what it's worth, ublox GPS modules and their clones should be configurable to always produce a PPS signal regardless of whether or not they have a satellite fix. The module would probably do a better job than software on a pi could during transient periods without a fix (due to how accurate the oscillators need to be in a GPS module). So, as long as you can trust the GPS module to exist and be powered, you should be able to reliably clock your display update with it. The only reason really to generate your own PPS would be if you want it to work without a GPS module at all, perhaps by NTP or something; you're then of course again looking at only a millisecond or so of accuracy.

Re: FPGA NTP Server

#68

Post by Unklejoe »

eqvinox wrote: post #66
mlyle wrote: post #65

Earlier quoted context omitted.

Yup-- I didn't know the prevalence so I didn't say. You're wrong though-- there's no ethernet adapter in my laptop ;)

Touché :D (sadly, 802.11 timestamping on wifi is very fringe...)

It's hard to do PTP reliably over WiFi since the symmetry in both directions (clientap) isn't really consistent. It _does_ sort of work though.

Re: FPGA NTP Server

#69

Post by willis936 »

sgtnoodle wrote: post #67
willis936 wrote: post #63

Earlier quoted context omitted.

Nixie tubes driven by a pair of cascaded HV5122 (driver + shift register). The strobe input is what updates the output registers with the recently shifted in contents. The driver takes 500 ns to turn on and the nixie tubes take about 10 us to fire once the voltage is applied. I know it's absurd to worry about the last few ms, but it's part of what interests me about the project. The goal is to make The Wall Time as a…

Sounds like fun! For what it's worth, ublox GPS modules and their clones should be configurable to always produce a PPS signal regardless of whether or not they have a satellite fix. The module would probably do a better job than software on a pi could during transient periods without a fix (due to how accurate the oscillators need to be in a GPS module). So, as long as you can trust the GPS module to exist and be po…

I'm using an uputronics GPS/RTC hat that has a u-blox M8 engine. I set it to stationary mode for extra accuracy. I'll have to look into other configuration options.

I'm currently trying to source PPS from software with this repo: https://github.com/twteamware/raspberrypi-ptp/blob/master/pp...

You might find this repo interesting.

I have a 500 MHz oscilloscope on hand so I can decently measure the offset between the raspberry pi PPS and GPS PPS.

Re: FPGA NTP Server

#70

Post by mlyle »

Unklejoe wrote: post #68
eqvinox wrote: post #66

Earlier quoted context omitted.

Touché :D (sadly, 802.11 timestamping on wifi is very fringe...)

It's hard to do PTP reliably over WiFi since the symmetry in both directions (client ap) isn't really consistent. It _does_ sort of work though.

It could be made to work fairly well with hardware / driver support. Timing information of frames is known rather precisely, and delay spread is <100ns in most environments.
Post reply on HN