Designing a Low Latency 10G Ethernet Core (2023)

Cool. We had a small company in San Diego called Metacomp (long gone now) design a Z8530 add on module for their 80186 based Multibus board. It had two Z8530 chips on it and you could have two modules on the base board. So 8 channels total per slot.

In fairness, assuming 4/8 ports in, 4 ports out, operating at some ungodly GHz using a custom GaAs or SiGe chip, and working in gearboxed scrambled space with very clever input mac prefix mapping and output scrambled/gearboxed precomputation, one _could_ do around 8ns optimistically from the start of the first 64b/66b block (the 0x55... preamble) for 10GB. There's some stuff about preamble shrink that makes it wonky, but that is doable. And interestingly enough, for 25G-R, one can comfortably do this in about 4ns. I am not in fact aware of such a beastie existing for 25G, but I have seen 1 or 2 for 10G though.

Surprisingly, if ChatGPT is prompted _juust_ right, it will even give you a good way to do this.

Yes, 8 was just about the maximum, simply because that would approach the limits of how many interrupts you could process before you'd inevitably start losing them. Also, space and power delivery limitations, and connector space on the backside made doing more than that very impractical. Or you had to take more than one interrupt per board but that was 'not done'. I've always loved working on that division line between hardware and software, as close to the metal as possible. Funny, I'd all but forgotten about this project, now I'm wondering if I still have the software somewhere, just searched for a bit and can't locate it, so there is a good chance I wrote it on company hardware. I did find another project that I'd forgotten about, a CAD program for sails for sailboats that I wrote in the 80's for TD Sails in the Netherlands. That was a great project to work on and I met some awesome people doing it.

Sorry but I don't understand any of your arguments.

You are throwing things like "HFT increase the barrier to entry", "HFT make things unfair", but... how so?

What would _you_ do if you had the capability to trade at the speed of light while others cannot?

Trading faster does not magically make money appear in your account, it does not magically tell you what is going up and what is going down.

It seems people work under the assumption that HFTs make money by having the same ideas than others, but trading it faster. That is _not_ what market making is, that is _not_ what HFTs do, and trading 10ns "faster" is of absolutely no use to 99.99% of market participants. In fact, the vast majority of market participants purposefully *slow* their signals to avoid impacting the market. At any given point in time, the orderbook contains but a tiny fraction of the liquidity that most people are willing to trade.

HFTs usually run on a combination of 4 businesses, none of which provide any downside for the rest of the participants: Market making, events signals, order flow and arbitrage.

Market making: essentially they are paid (or more precisely, rebated) to "animate" the market (provide liquidity). Everyone is happier with tight spreads, and everyone is happier with some level of liquidity at the top of the book, I don't really see how that could be up to debate. This is a tough business, the market maker has to provide a contractual amount of inventory for buyers and sellers at the same time, with the risk that these assets can inadvertently increase or decrease in value. Market makers get a premium (the spread) in exchange for providing liquidity to market takers, taking the risk of holding that inventory for them.

Order flow: HFTs are usually more knowledgeable than managers (and even brokers) to perform trade execution. Remember, the price of an asset does not move "by itself". What makes the price of an asset _really_ increase is an imbalance between the size of buys and sells, causing the spread to be crossed increasingly in one direction, and eating higher and higher price levels => a market participant buying shares is self realizing the price increase. That is called market impact, and that is something nobody wants. In an ideal world, nobody would know the actual liquidity that other participants are willing to exchange on the market, that way no arbitrage would be possible, no market impact, everyone is happy. The problem though is that you need _some_ level of visibility otherwise you hinder so called "price discovery", and regulators don't like that because it's an open door for scams, hidden fees, etc. So a lot of investors just delegate the actual order passing to HFTs: you agree on a price you're happy with, and the HFT executes the orders for you. If they manage to get a better price: good for them, they can pocket the difference. If they didn't, they will have to eat the cost and do better next time. Again, it's a _difficult job_, because by very definition managers are going to try to give you _toxic flows_: if someone mandates you to buy 100 shares of Microsoft, it's because they think the price will go up, but you will quote them at the current stock price. I don't think anyone is negatively impacted by that business to be honest. I am quite happy to instantly do a trade at a known price and let an HFT take the risk of executing trades, finding liquidity, etc.

Arbitrage: This may be the most well known HFT strategy. Essentially benefiting from price disparity of more or less fungible assets. So if HSBC is priced the equivalent of $100 on the LSE and $101 on NYSE, they buy there and sell here. I don't think there is much debate on whether that's a good thing or not, everyone should be happy that prices are more or less consistent around the world. Most people also wouldn't want to bear the overnight nor settlement/FX risk of holding these.

Event signals: I guess that's what people overreact to, even though this is but a tiny and negligible part of the HFT businesses. There are strategies where the whole concept is just to be the first to react to some kind of event, and yes HFTs are the best placed to do such strategies. The thing is, this is not a very lucrative business for a simple reason, which I already stated before: at any point in time, the orderbook contains but a mere fraction of the available liquidity. And it takes *ample time* for the market impact of a trade to be absorbed. There are a lot of papers on the subject, but without entering in too much details: the bigger your order, the longer it will take for the book to absorb it and liquidity to come back. For most hedge funds doing statistical arbitrage, CTAs, etc, we are talking multiple *days*. So realistically, if you're in the business of reacting the fastest to some kind of news or event, there's really not much more you can do than cross the spread and eat one or two levels of the book. Even for the most liquid of US stocks, that's a couple hundred thousands dollar per trade on average, for a tiny return difference versus executing a couple of seconds later, so I hope you find a lot of these if you plan on getting rich.

TLDR, I think 99% of HFT hate is completely unjustified. They provide a legitimate service to most market participants, and as an investor I wouldn't want to do their job, nor would I think that they hinder in any way my investments, on the contrary.

actually you do need another participant to take the other side of your trade. That’s not a middle man. Market makers, liquidity providers, are a key component of these markets, taking risk with their own capital (no systemic risk, no too big to fail, going to get a government is things go badly), requiring sophistication and specialization to survive in an ultra competitive environment with a high degree of uncertainty and risk exposure.