Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Designing a Low Latency 10G Ethernet Core (2023)

ttchisholm.github.io

Post reply on HN
Print view View original

41–50 of 83 posts

Re: Designing a Low Latency 10G Ethernet Core (2023)

#41

Post by amelius »

jbstack wrote: post #39
amelius wrote: post #33

Earlier quoted context omitted.

> It seems fun to be a high frequency FPGA trader designer. Yes, if you think that doing pointless work is fun. What do you tell your grandchildren?

It's a bit unfair to call it pointless. Empirical studies have shown that HFT increase liquidity in markets, leading to narrowed spreads, market depth, faster execution speeds, cross-market informational efficiency, etc. All of these are generally beneficial to other non-HFT market participants. Now you might counter that it also brings some negatives (e.g. fleeting liquidity, market manipulation risk, etc.) and ther…

> Also, it's not pointless to the person actually engaging in HFT if they are making money from it. Activities aren't pointless if they pay your rent, put food on your table, etc.

You have to look at the bigger picture. HFT companies only increase the barrier to entry to participate in the market. This means that while you may be able to put food on the table, you make it more difficult for other people to put food on the table.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#42

Post by jbstack »

amelius wrote: post #41
jbstack wrote: post #39

Earlier quoted context omitted.

It's a bit unfair to call it pointless. Empirical studies have shown that HFT increase liquidity in markets, leading to narrowed spreads, market depth, faster execution speeds, cross-market informational efficiency, etc. All of these are generally beneficial to other non-HFT market participants. Now you might counter that it also brings some negatives (e.g. fleeting liquidity, market manipulation risk, etc.) and ther…

> Also, it's not pointless to the person actually engaging in HFT if they are making money from it. Activities aren't pointless if they pay your rent, put food on your table, etc. You have to look at the bigger picture. HFT companies only increase the barrier to entry to participate in the market. This means that while you may be able to put food on the table, you make it more difficult for other people to put food o…

How's that? HFT lowers the barrier of entry for ordinary participants who just want to invest their savings either directly or through something like a pension fund, because they get all those benefits I mentioned, and those benefits are useful when you are only executing a trade once a month or year.

It only increases the barrier to entry to people who want to do HFT themselves on a smaller scale, because they now can't compete on latency etc. But it's a bit of a circular argument to say that it's a bad thing that engaging in HFT increases the barrier of entry to HFT.

In any case, and to reiterate, I wasn't refuting a claim that HFT is a bad thing, I was refuting your original claim that it's pointless. For something to be pointless, there has to be no purpose to it whatsoever, which means it has to have no benefits.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#43

Post by jacquesm »

drmpeg wrote: post #38
jacquesm wrote: post #29

Earlier quoted context omitted.

Shit happens. I wrote the firmware for a multi-link HDLC card which earned vast amounts of money for the company that commissioned it. I was quite aware of my responsibility and so was the hardware team. We stress tested the crap out of it before releasing to production and fortunately it never locked up when it was in actual use but we had some pretty wild and very rare (so hard to trigger and reproduce) bugs which…

Multi-link HDLC. That brings me back to 1985 and the Intel 8274, Zilog Z8530 and the Western Digital WD2511 (that implemented the layer 2 protocol in silicon).

Yes, that was around that time. There were a couple of problems, the first batch of chips we got was very early in the development stage of the chip, pre-production issues and there were some bugs in the chips themselves which could cause lock-up under some circumstances. We found ways to work around those and then of course there were all of the niceties around dealing with a device that generates an extremely high rate of interrupts. So the code would either have to attempt to service all channels on any interrupt or be re-entrant. I don't remember which solution we picked but in the end the thing was, once we had the bugs worked out fairly bullet proof.

One funny bit about the development process was that initially I was going to be nice and implement every layer as its own stand-alone bit of software communicating via a defined set of primitives with the layers above and below. But that was slow as molasses so in the end all of that elegance got discarded for an absolutely unholy sandwich that did all of the layers in a single chunk of code. But with the experience from the 'slow' version that was actually doable, I would have never been able to write that as the first implementation. Classic illustration of 'first make it work, then make it fast'.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#44

Post by OneDeuxTriSeiGo »

amelius wrote: post #40
le-mark wrote: post #34

Earlier quoted context omitted.

High frequency trading is far from the only pointless activity software and hardware developers are employed to do. At least in that realm they are pushing the state of the art, unlike presenting ads or yet another crud app.

Pushing the state of the art? That's only meaningful if the results are published. But I suspect that everything is kept secret. Come on, it is not difficult to find better things to do with your brains. Like work in medicine, etc.

You do realise that Jane Street (one of the big HFT market makers) publishes basically all their technical work and contributes to massive amounts of open source software? They are to my knowledge one of the main contributors to the OCaml ecosystem as well.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#45

Post by foobar10000 »

mgrosvenor wrote: post #36
mrlongroots wrote: post #14

Earlier quoted context omitted.

The other funny bit is that one-way PCIe latency is 250ns-ish (don't quote me on the exact numbers), which imposes a hard 1us constraint on latency between two hosts.

You can go quicker with CXL, but not by much.

There are always DPUs like NextSilicon, NVidia's Blue*, etc - at under 100 ns of SoF to fast compute though.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#46

Post by OneDeuxTriSeiGo »

amelius wrote: post #41
jbstack wrote: post #39

Earlier quoted context omitted.

It's a bit unfair to call it pointless. Empirical studies have shown that HFT increase liquidity in markets, leading to narrowed spreads, market depth, faster execution speeds, cross-market informational efficiency, etc. All of these are generally beneficial to other non-HFT market participants. Now you might counter that it also brings some negatives (e.g. fleeting liquidity, market manipulation risk, etc.) and ther…

> Also, it's not pointless to the person actually engaging in HFT if they are making money from it. Activities aren't pointless if they pay your rent, put food on your table, etc. You have to look at the bigger picture. HFT companies only increase the barrier to entry to participate in the market. This means that while you may be able to put food on the table, you make it more difficult for other people to put food o…

Markets should not primarily be a casino.

If you want to invest some of your income into a spread of companies? Go for it. But if all you are doing is trying to game the market? That's just gambling, go to a real casino and have some fun instead.

HFT should be driven by these types of market makers exactly for the purpose of decreasing the amount of unnecessary speculation and volatility in the markets. HFTs killing the spreads is one of the only reasons the markets didn't implode during covid despite the extraordinary fear and volatility that was impacting them.

You want a parasite? Look to hedge funds who just sit on piles of other people's cash to scoop obscene amounts off the top (relative to their actual contribution) for occassionally shuffling funds between financial instruments.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#47

Post by amelius »

jbstack wrote: post #42
amelius wrote: post #41

Earlier quoted context omitted.

> Also, it's not pointless to the person actually engaging in HFT if they are making money from it. Activities aren't pointless if they pay your rent, put food on your table, etc. You have to look at the bigger picture. HFT companies only increase the barrier to entry to participate in the market. This means that while you may be able to put food on the table, you make it more difficult for other people to put food o…

How's that? HFT lowers the barrier of entry for ordinary participants who just want to invest their savings either directly or through something like a pension fund, because they get all those benefits I mentioned, and those benefits are useful when you are only executing a trade once a month or year. It only increases the barrier to entry to people who want to do HFT themselves on a smaller scale, because they now c…

We all know that middlemen are bad.

We don't need an HFT middleman.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#48

Post by [deleted] »

amelius wrote: post #33
Neywiny wrote: post #2

It seems fun to be a high frequency FPGA trader designer. All my FPGA is much lower power consumption so I don't get to play with stuff like gigs of external SRAM or the QDR stuff or whatnot

> It seems fun to be a high frequency FPGA trader designer. Yes, if you think that doing pointless work is fun. What do you tell your grandchildren?

[deleted]

Re: Designing a Low Latency 10G Ethernet Core (2023)

#49

Post by amelius »

OneDeuxTriSeiGo wrote: post #46
amelius wrote: post #41

Earlier quoted context omitted.

> Also, it's not pointless to the person actually engaging in HFT if they are making money from it. Activities aren't pointless if they pay your rent, put food on your table, etc. You have to look at the bigger picture. HFT companies only increase the barrier to entry to participate in the market. This means that while you may be able to put food on the table, you make it more difficult for other people to put food o…

Markets should not primarily be a casino. If you want to invest some of your income into a spread of companies? Go for it. But if all you are doing is trying to game the market? That's just gambling, go to a real casino and have some fun instead. HFT should be driven by these types of market makers exactly for the purpose of decreasing the amount of unnecessary speculation and volatility in the markets. HFTs killing…

> HFTs killing the spreads is one of the only reasons the markets didn't implode during covid despite the extraordinary fear and volatility that was impacting them.

And why would this not be possible if everybody traded at the same, low, frequency? Alternatively, what is the minimum trading frequency required to save markets from collapsing?

By trying to answer these questions you see how ridiculous the situation with HFT really is.

Re: Designing a Low Latency 10G Ethernet Core (2023)

#50

Post by foobar10000 »

rrr_eady wrote: post #37
mgrosvenor wrote: post #35

Earlier quoted context omitted.

No they don’t. At least not on the critical path. L1 switch’s at 4ns simply replicate the incoming electrical signal. That’s all they do. They also do some packet accounting and cute features off the critical path. None of that at 4ns.

This guy knows a thing or two about these devices ;)

Yes he very much does :)
Post reply on HN