Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Barbarians at the Gateways: High-frequency Trading and Exchange Technology

queue.acm.org

Post reply on HN
Print view View original

51–60 of 146 posts

Re: Barbarians at the Gateways: High-frequency Trading and Exchange Technology

#51

Post by dclusin »

rdtsc wrote: post #49
dclusin wrote: post #7

Earlier quoted context omitted.

Kernel bypass is also common. Having the OS manage your networking stack adds too much latency.

That is the most common one I've heard of. Obviously if they are building fpgas into the switch itself then well of course they'd already tried running customized ethernet driver firmware and all.

It really depends on what asset class you are trading, like the author mentioned. For equities the performance demands are out of control (< 50 micros). For FX, it's still pretty tight but much more manageable.

Re: Barbarians at the Gateways: High-frequency Trading and Exchange Technology

#52

Post by 31reasons »

dopamean wrote: post #32
cs702 wrote: post #27

So much talent... focused on the buying and selling of securities , instead of creating new things that will make the world better in a directly measurable manner. Virtually all trading volume today consists of buying and selling old securities -- essentially, legal claims on existing assets. The sale of new securities issued to finance the creation of new products and services -- for example, a company selling new s…

Someone said something similar about a lot of people in the startup community. That we were sending our best and brightest out to create products that ultimately only served the purpose of pushing ads (Instagram, Facebook, Twitter, etc). I don't know how true this is. There are a lot of very smart people in the world. The luxury of having that many smart people is that we can have a lot dedicated to designing amazing…

Then why we haven't cured cancer yet or malaria or global warming or psychological disorders ? There are a lot of smart people in the world but we really can't afford to work on silly things.

Re: Barbarians at the Gateways: High-frequency Trading and Exchange Technology

#53

Post by kylebrown »

The sub-penny rule (decimalization) was also mentioned by Chris Stucchio (aka yummyfajitas).[1] And it is extensively discussed in the book "Dark Pools".[2] Because order queues only exist at whole-penny prices, the bots are forced to compete on speed. If it weren't for that rule, there would also be competition on price because you could place offers and get price-priority in the sub-penny interval.

The playing field is basically a DDoS arms race, where the main technique is full-throttle quote-stuffing to push anyone without equivalent hardware and connection away from the front of the queue, and out of the game. (Dark Pools also mentions a conspiracy of special undocumented order types on certain exchanges). The ACM article even admits that the algos aren't even particularly intelligent, just very speedy: "In my experience, most high-frequency algorithms are fairly straightforward in concept—but their success is based largely on how quickly they can interact."

A distributed order book like the one in the Ripple protocol[3] is a much fairer system. The network charges a transaction fee in order to prevent quote spam/DoS attacks, price fractions can be as small as 6 decimal places, and offers are processed in an effectively random order (within each discrete ledger).[4]

1. http://www.chrisstucchio.com/blog/2012/hft_whats_broken.html

2. Dark Pools: The Rise of the Machine Traders and the Rigging of the U.S. Stock Market by Scott Patterson

3. https://ripple.com/distributed-fx/

4. https://ripple.com/wiki/Arbitrage#Rapidly_changing_offers

Re: Barbarians at the Gateways: High-frequency Trading and Exchange Technology

#54

Post by rfnslyr »

chollida1 wrote: post #6

> Some facilities such as the Mahwah, New Jersey, NYSE (New York Stock Exchange) data center have rolls of fiber so that every cage has exactly the same length of fiber running to the exchange cages.3 I remember the first time someone told me that. I thought they were kidding. Then I actually got to see the data center. The exchanges, and HFT firms, take this very seriously. The speed at which they execute is just un…

How do I get into this field? I'm a 20 year old making a shit ton of money by doing mobile work but I find it so boring. Nothing about it is innovative. I feel like a high level computer construction worker. It's brainless for the most part, it just takes time.

I want to get into some really cool stuff. I've been wanting to get into investment banking from a technical standpoint, HFT, algorithmic trading etc, but honestly don't know where to start.

What are the de facto books/websites/reads for this field?

Re: Barbarians at the Gateways: High-frequency Trading and Exchange Technology

#55

Post by anonymoushn »

tucson wrote: post #22

""It's legalized front-running. I think it is basically evil and I don't think it should have ever been allowed to reach the size that it did," he said. "Why should all of us pay a little group of people to engage in legalized front-running of our orders?"" Charlie Munger, http://www.cnbc.com/id/100705820

Which HFT firms have access to the orders of other participants before the exchange does?

Re: Barbarians at the Gateways: High-frequency Trading and Exchange Technology

#56

Post by socillion »

foobarian wrote: post #30
cs702 wrote: post #27

So much talent... focused on the buying and selling of securities , instead of creating new things that will make the world better in a directly measurable manner. Virtually all trading volume today consists of buying and selling old securities -- essentially, legal claims on existing assets. The sale of new securities issued to finance the creation of new products and services -- for example, a company selling new s…

I always wondered whether it would make sense to legislate a random added latency in exchanges in order to create a latency noise floor to remove the need for HFT. Without such a cap it's an arms race that adds no value.

> Without such a cap it's an arms race that adds no value.

I have seen nobody ever challenge the value of liquidity in these discussions of HFT. Assuming this, the argument is that at some amount of latency the additional liquidity turns into a negative.

It seems more like the additional value is simply extremely small, but meaningful over an entire market.

What's the exact cutoff point for the value of additional liquidity turning negative, and will anyone agree on that arbitrary number?

Assuming HFT does turn into a negative at some point, wouldn't the same apply to optimizations in other markets like online advertising?

If it does apply elsewhere, we should also legislate away the practice of making online advertising more efficient to prevent tech giants like Google and Facebook from wasting billions of dollars and untold man-hours on these activities that are clearly worthless to society.

Re: Barbarians at the Gateways: High-frequency Trading and Exchange Technology

#57

Post by anonymoushn »

beagle3 wrote: post #31
Mikeb85 wrote: post #28

Earlier quoted context omitted.

HFT exploits arbitrage opportunities. This reduces spreads, leading to better execution for everyone.

Ask any HFT person, and they'll tell you less than 1% of their profit is from arbitrage. Unless, of course, you include the practice of quote stuffing (Which causally generates a latency arbitrage against slower players) as part of "arbitrage" - but that's not much different than saying a robber exploits the arbitrage between the fear for your life and willingness to part with the money on your person.

I can confirm that more than 1% is from arbitrage, and that we do not engage in quote stuffing/flickering/etc.

Re: Barbarians at the Gateways: High-frequency Trading and Exchange Technology

#58

Post by jrehor »

dopamean wrote: post #8

I'm wondering if someone here can explain to me the value in HFT for anyone other than the people doing the trading. I went to Wharton and some old friends are in HFT and the one thing they all say, and I hear repeated everywhere, is that they provide liquidity to the market. Something about it just doesn't ring true to me. If it is in fact true I'd love to hear an explanation.

You are assuming that HFT profits come at the expense of investors. In fact, they were taken from insiders.

I'm old enough to remember trading on US stock exchanges in the mid 1990s when prices where quoted in 1/8s and 1/16s and NYSE specialists were the only ones with visibility into order book. Think about it: you as an investor had no idea of the depth of the order book, but the specialist who took the other side of the trade from you had it in front of him. The specialists were minting money. They would lose money maybe one day per quarter, and their ROEs were ludicrous.

HFT and ECN trading killed them. Labranche, Van der Moolen, Susquehanna, Spear Leeds, all gone. Goldman Sachs bought Spear Leeds for $6 billion in 2000 (it is now no more); Labranche specialist business fetched only $25m when it was sold to Barclays in 2010. Van der Moolen went bankrupt in 2010.

Investment bank trading desks, true champions of customer front-running, are shrinking fast. Cash equity trading has become so tough for the banks that they are starting to think of it as a cost center, a loss leader to promote their equity underwriting business.

These were multi-billion dollar businesses, with tens of thousands of middlemen living high on the hog from the spreads and front-running. They're all (mostly) gone. Good riddance.

If you want to relive the old days of trading before HFT, go execute a large trade on Karachi Stock Exchange. Put the order in and watch in amazement.

I would like to see the data showing total revenues of specialists, market makers, bank trading desks, brokerages, and HFT traders, over time. I would bet they have been going down for two decades. This is undeniably a good thing for investors.

Re: Barbarians at the Gateways: High-frequency Trading and Exchange Technology

#59

Post by pdovy »

cs702 wrote: post #27

So much talent... focused on the buying and selling of securities , instead of creating new things that will make the world better in a directly measurable manner. Virtually all trading volume today consists of buying and selling old securities -- essentially, legal claims on existing assets. The sale of new securities issued to finance the creation of new products and services -- for example, a company selling new s…

Virtually all trading volume today consists of buying and selling old securities -- essentially, legal claims on existing assets. The sale of new securities issued to finance the creation of new products and services -- for example, a company selling new shares via an IPO -- represents only a minuscule portion of total trading volume.

--

Leaving aside for a moment the question of the value of HFT, this is a skewed view of the world of finance. The equities markets may be the most visible to the public (quoted on the news, associated most directly with companies and products we all know, etc), but it's just one market.

In fact, the size of global bond markets is almost double that of equities [1] and is a huge source of funding for companies (and governments), and has more frequent new issues for the same entity than in equities. Similarly futures markets are hugely active and important as both a source of price discovery for commodities and as a way for businesses small and large to hedge risk.

I mention all this not to pick on you in particular, but because it seems in the wake of the financial crises of the past few years, this "finance is bad" mentality seems to be pervasive. There are absolutely problems, but I think it's important not to lose sight of the fact that much of what financial markets do is hugely important to modern businesses and governments.

[1] http://finance.zacks.com/bond-market-size-vs-stock-market-si...

Re: Barbarians at the Gateways: High-frequency Trading and Exchange Technology

#60

Post by kylebrown »

31reasons wrote: post #52
dopamean wrote: post #32

Earlier quoted context omitted.

Someone said something similar about a lot of people in the startup community. That we were sending our best and brightest out to create products that ultimately only served the purpose of pushing ads (Instagram, Facebook, Twitter, etc). I don't know how true this is. There are a lot of very smart people in the world. The luxury of having that many smart people is that we can have a lot dedicated to designing amazing…

Then why we haven't cured cancer yet or malaria or global warming or psychological disorders ? There are a lot of smart people in the world but we really can't afford to work on silly things.

Price discovery in markets, and economic boom-bust cycles are a societal/tech problem going back hundreds if not thousands of years. HFT is only part of the solution (or should be), and I don't think its silly. And even if it were, sometimes there's just no getting around the silly/thorny issues (makefiles anyone?) before you work on the important stuff directly.
Post reply on HN