Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Designing a Low Latency 10G Ethernet Core (2023)

ttchisholm.github.io

Post reply on HN
Print view View original

31–40 of 83 posts

Re: Designing a Low Latency 10G Ethernet Core (2023)

#31

Post by darrin »

userbinator wrote: post #9

less than 60ns loopback latency There are some Ethernet switches with 4ns latency, and those do more than just sending and receiving, so there's clearly still an order of magnitude of improvement still available. 4ns is basically ~40 cycles of the bit clock for 10G Ethernet.

There are L1 crosspoint "switches" in the 2-5ns range depending on port density and similar modes in some Ethernet silicon. These are not Ethernet switches in any normal sense though. They only replicate the signal 1:1 or 1:n and do not dynamically switch the destination based on anything in the packet. The fastest L3 Ethernet switches on the market are ~90ns.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#32

Post by exogenousdata »

anthonj wrote: post #11
throwaway2037 wrote: post #10

LinkedIn tells me: https://uk.linkedin.com/in/ttchisholm FPGA engineer with a focus on ultra-low latency networking at Jane Street. Yikes.

Not sure i get the implication, something wrong with the company?

I’m not the parent, but I’d assume that this was an impressed ‘yikes.’ While Jane is not the most well known firm for ultra low latency trading, an FPGA engineer from that firm could be considered an expert practitioner in the field.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#33

Post by amelius »

Neywiny wrote: post #2

It seems fun to be a high frequency FPGA trader designer. All my FPGA is much lower power consumption so I don't get to play with stuff like gigs of external SRAM or the QDR stuff or whatnot

> It seems fun to be a high frequency FPGA trader designer.

Yes, if you think that doing pointless work is fun. What do you tell your grandchildren?

Re: Designing a Low Latency 10G Ethernet Core (2023)

#34

Post by le-mark »

amelius wrote: post #33
Neywiny wrote: post #2

It seems fun to be a high frequency FPGA trader designer. All my FPGA is much lower power consumption so I don't get to play with stuff like gigs of external SRAM or the QDR stuff or whatnot

> It seems fun to be a high frequency FPGA trader designer. Yes, if you think that doing pointless work is fun. What do you tell your grandchildren?

High frequency trading is far from the only pointless activity software and hardware developers are employed to do. At least in that realm they are pushing the state of the art, unlike presenting ads or yet another crud app.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#35

Post by mgrosvenor »

userbinator wrote: post #9

less than 60ns loopback latency There are some Ethernet switches with 4ns latency, and those do more than just sending and receiving, so there's clearly still an order of magnitude of improvement still available. 4ns is basically ~40 cycles of the bit clock for 10G Ethernet.

No they don’t. At least not on the critical path. L1 switch’s at 4ns simply replicate the incoming electrical signal. That’s all they do.

They also do some packet accounting and cute features off the critical path. None of that at 4ns.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#36

Post by mgrosvenor »

mrlongroots wrote: post #14
userbinator wrote: post #9

less than 60ns loopback latency There are some Ethernet switches with 4ns latency, and those do more than just sending and receiving, so there's clearly still an order of magnitude of improvement still available. 4ns is basically ~40 cycles of the bit clock for 10G Ethernet.

The other funny bit is that one-way PCIe latency is 250ns-ish (don't quote me on the exact numbers), which imposes a hard 1us constraint on latency between two hosts.

You can go quicker with CXL, but not by much.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#37

Post by rrr_eady »

mgrosvenor wrote: post #35
userbinator wrote: post #9

less than 60ns loopback latency There are some Ethernet switches with 4ns latency, and those do more than just sending and receiving, so there's clearly still an order of magnitude of improvement still available. 4ns is basically ~40 cycles of the bit clock for 10G Ethernet.

No they don’t. At least not on the critical path. L1 switch’s at 4ns simply replicate the incoming electrical signal. That’s all they do. They also do some packet accounting and cute features off the critical path. None of that at 4ns.

This guy knows a thing or two about these devices ;)

Re: Designing a Low Latency 10G Ethernet Core (2023)

#38

Post by drmpeg »

jacquesm wrote: post #29
ktm5j wrote: post #27

Earlier quoted context omitted.

What about when something goes wrong because you overlooked some detail?

Shit happens. I wrote the firmware for a multi-link HDLC card which earned vast amounts of money for the company that commissioned it. I was quite aware of my responsibility and so was the hardware team. We stress tested the crap out of it before releasing to production and fortunately it never locked up when it was in actual use but we had some pretty wild and very rare (so hard to trigger and reproduce) bugs which…

Multi-link HDLC. That brings me back to 1985 and the Intel 8274, Zilog Z8530 and the Western Digital WD2511 (that implemented the layer 2 protocol in silicon).

Re: Designing a Low Latency 10G Ethernet Core (2023)

#39

Post by jbstack »

amelius wrote: post #33
Neywiny wrote: post #2

It seems fun to be a high frequency FPGA trader designer. All my FPGA is much lower power consumption so I don't get to play with stuff like gigs of external SRAM or the QDR stuff or whatnot

> It seems fun to be a high frequency FPGA trader designer. Yes, if you think that doing pointless work is fun. What do you tell your grandchildren?

It's a bit unfair to call it pointless. Empirical studies have shown that HFT increase liquidity in markets, leading to narrowed spreads, market depth, faster execution speeds, cross-market informational efficiency, etc. All of these are generally beneficial to other non-HFT market participants.

Now you might counter that it also brings some negatives (e.g. fleeting liquidity, market manipulation risk, etc.) and there's certainty some validity to that. But none of this supports an argument that it's pointless.

More generally, activities aren't pointless if you are earning money from it which pays your rent, puts food on your table, etc. A large portion of the world works mainly for this goal. It's perfectly valid to tell your grandchildren "I spent my life making sure we'd have enough money for you and your parents to live comfortably", and we shouldn't look down on someone who does that.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#40

Post by amelius »

le-mark wrote: post #34
amelius wrote: post #33

Earlier quoted context omitted.

> It seems fun to be a high frequency FPGA trader designer. Yes, if you think that doing pointless work is fun. What do you tell your grandchildren?

High frequency trading is far from the only pointless activity software and hardware developers are employed to do. At least in that realm they are pushing the state of the art, unlike presenting ads or yet another crud app.

Pushing the state of the art? That's only meaningful if the results are published. But I suspect that everything is kept secret.

Come on, it is not difficult to find better things to do with your brains. Like work in medicine, etc.

Post reply on HN