Live data from Hacker News

Designing a Low Latency 10G Ethernet Core (2023)

ttchisholm.github.io

21–30 of 83 posts

Re: Designing a Low Latency 10G Ethernet Core (2023)

#21
post #13
post #3

Earlier quoted context omitted.

I feel like it's also got to be super stressful Imagine getting a call you system is down and is costing them millions per hour

I'd want to know why they had such a system as a single point of failure

No business is designed ideally. Now do you want the job or we move to another one who will accept this?

Re: Designing a Low Latency 10G Ethernet Core (2023)

#22

less than 60ns loopback latency There are some Ethernet switches with 4ns latency, and those do more than just sending and receiving, so there's clearly still an order of magnitude of improvement still available. 4ns is basically ~40 cycles of the bit clock for 10G Ethernet.

For switching, this is just not true. There's in/out solutions that are at this kind of scale, but they are by definition not implementing any switching logic.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#23
post #3
post #2

It seems fun to be a high frequency FPGA trader designer. All my FPGA is much lower power consumption so I don't get to play with stuff like gigs of external SRAM or the QDR stuff or whatnot

I feel like it's also got to be super stressful Imagine getting a call you system is down and is costing them millions per hour

The way to think is the opposite: your system is making millions per hour.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#24
post #20
post #15

Earlier quoted context omitted.

Cut Through switching needs what, 14 bytes? Thats ~12ns on 10G? How would you do it in 40 cycles?

why do you assume the full ethernet header? IIRC. cut-through only needs the first 6 bytes. Since it only needs the destination address for the port lookup. Potentially the first bit, on broadcast.

Because you might want VLAN. Plus you cant just start blasting reply without a preamble, so its still 14 bytes after receiving just the dest MAC. Then you get potential IFG, FEC, scrambling, it all adds up, no way any switch can do 4ns without heavy lawyer talk in the small print.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#25
post #23
post #3

Earlier quoted context omitted.

I feel like it's also got to be super stressful Imagine getting a call you system is down and is costing them millions per hour

The way to think is the opposite: your system is making millions per hour.

No the way to think is ‘it’s down less than our closest competitors system’

Re: Designing a Low Latency 10G Ethernet Core (2023)

#26
post #13
post #3

Earlier quoted context omitted.

I feel like it's also got to be super stressful Imagine getting a call you system is down and is costing them millions per hour

I'd want to know why they had such a system as a single point of failure

Failure modes can be a bug making a bad trade while doing so at 100% uptime.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#27
post #23
post #3

Earlier quoted context omitted.

I feel like it's also got to be super stressful Imagine getting a call you system is down and is costing them millions per hour

The way to think is the opposite: your system is making millions per hour.

What about when something goes wrong because you overlooked some detail?

Re: Designing a Low Latency 10G Ethernet Core (2023)

#29
post #27
post #23

Earlier quoted context omitted.

The way to think is the opposite: your system is making millions per hour.

What about when something goes wrong because you overlooked some detail?

Shit happens. I wrote the firmware for a multi-link HDLC card which earned vast amounts of money for the company that commissioned it. I was quite aware of my responsibility and so was the hardware team. We stress tested the crap out of it before releasing to production and fortunately it never locked up when it was in actual use but we had some pretty wild and very rare (so hard to trigger and reproduce) bugs which delayed deployment considerably. But none of this gave me the kind of feeling that working on fuel estimation software for aircraft gave. That's when there are in the most literal sense lives on the line, and that's a completely different kind of pressure. You simply can not fuck up. It also really rammed home the value of code review and having a good specification so that you can ensure that within the envelope of input parameters your software does what it is supposed to do.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#30

less than 60ns loopback latency There are some Ethernet switches with 4ns latency, and those do more than just sending and receiving, so there's clearly still an order of magnitude of improvement still available. 4ns is basically ~40 cycles of the bit clock for 10G Ethernet.

It's a bit more complicated than that. Loopback implies that the data went off the wire and was sent back. That puts some serious constraints on the process that deals with that data, how it is stored and how it is fed back into the system. You can't just compare that with naked pass-through latency (which is what a switch would do), because the signal is already on the way out before it has even fully been received.

Essentially a few ns after the header is received it will already be passed on to the output port after some signal conditioning, which happens almost without latency. A typical 'high speed' switch will have 800 ns round trip, so that 60 ns quoted here is actually quite impressive.

Post reply on HN