Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Knightmare: A DevOps Cautionary Tale (2014)

dougseven.com

Post reply on HN
Print view View original

21–30 of 34 posts

Re: Knightmare: A DevOps Cautionary Tale (2014)

#21

Post by manwithplan »

alasdair_ wrote: post #20
manwithplan wrote: post #19

Earlier quoted context omitted.

Cool story, didn't happen. There were no retail trading accounts at Knight. In fact, there was no outside money of any kind. The S&P500 fell about 0.75% on the day in question: a non-trivial decline, but not really remarkable. It was up about 0.4% on the week. Also, this is incredibly not how the OCC deals with members in default.

>There were no retail trading accounts at Knight. The article states “The NYSE was planning to launch a new Retail Liquidity Program (a program meant to provide improved pricing to retail investors through retail brokers, like Knight)” This pretty strongly implies Knight was a retail broker. I assume I’m missing something- can you clarify?

I don't know precisely what the article means. The NYSE Retail Liquidity Program, which still exists, describes two categories of participants: member organizations (MOs) and liquidity providers (LPs). MOs have retail orders, which are defined as originating with an actual person, ie not a computer. LPs provide liquidity to those orders, ie they take the other side of the trade.

I believe Knight would have been interested as an LP. It is not inconceivable that in some circumstances Knight would have been able to submit retail flow as an MO, but 100% of that flow would have been routed to it from brokerages holding actual retail accounts.

Knight was a trading firm, not a hedge fund, and certainly not an institution which held outside money in retail accounts. But consider an entity which does have retail accounts and also has propriety trading for its own account. Suggesting that the former would become inaccessible if the latter lost lots of capital in bad trading is absurd, would mean that retail-customer and proprietary monies were mingled, and would require the violation of untold numbers of regulations. This did not happen with Knight and indeed has never, ever happened.

Re: Knightmare: A DevOps Cautionary Tale (2014)

#22

Post by JackFr »

manwithplan wrote: post #19
JackFr wrote: post #13

Amusing personal anecdote -- the Knight debacle caused the market in general to tumble. The week before a coworker of mine -- sure of a market drop but for other reasons -- had bought a raft or puts on the S&P 500. When I saw looking glum at work after the Knight news broke, I asked him what was wrong didn't you make a ton? Yeah, he said, but I can't get out cause my account's with Knight.

Cool story, didn't happen. There were no retail trading accounts at Knight. In fact, there was no outside money of any kind. The S&P500 fell about 0.75% on the day in question: a non-trivial decline, but not really remarkable. It was up about 0.4% on the week. Also, this is incredibly not how the OCC deals with members in default.

This literally happened as I described it.

That is to say I have repeated what I was told. And it’s funny — I was all set to go to battle stations over this: 1) The close is not the same as how bad it was intraday. 2) Yes, the SIPC and CFTC have controls and he was able to access his account eventually after the profit opportunity was gone. 3) He was a sophisticated investor, and if Knight had retail accounts he might have been with them.

But in retrospect it’s too clever - it’s much more likely in my estimation that the dude in question with whom I worked was simply full of shit. He tells a bad beat story and it’s not like any of us asked to see statements. It never even occurred to me to doubt it until now.

Re: Knightmare: A DevOps Cautionary Tale (2014)

#23

Post by manwithplan »

JackFr wrote: post #22
manwithplan wrote: post #19

Earlier quoted context omitted.

Cool story, didn't happen. There were no retail trading accounts at Knight. In fact, there was no outside money of any kind. The S&P500 fell about 0.75% on the day in question: a non-trivial decline, but not really remarkable. It was up about 0.4% on the week. Also, this is incredibly not how the OCC deals with members in default.

This literally happened as I described it. That is to say I have repeated what I was told. And it’s funny — I was all set to go to battle stations over this: 1) The close is not the same as how bad it was intraday. 2) Yes, the SIPC and CFTC have controls and he was able to access his account eventually after the profit opportunity was gone. 3) He was a sophisticated investor, and if Knight had retail accounts he migh…

The Knight trading misbehavior bid the market up, not down. It didn't make its intraday lows until about 12:30 and the S&P500 never fell by more than 1%. Maybe if the puts were on Knight itself ...

Re: Knightmare: A DevOps Cautionary Tale (2014)

#24

Post by JackFr »

manwithplan wrote: post #21
alasdair_ wrote: post #20

Earlier quoted context omitted.

>There were no retail trading accounts at Knight. The article states “The NYSE was planning to launch a new Retail Liquidity Program (a program meant to provide improved pricing to retail investors through retail brokers, like Knight)” This pretty strongly implies Knight was a retail broker. I assume I’m missing something- can you clarify?

I don't know precisely what the article means. The NYSE Retail Liquidity Program, which still exists, describes two categories of participants: member organizations (MOs) and liquidity providers (LPs). MOs have retail orders, which are defined as originating with an actual person, ie not a computer. LPs provide liquidity to those orders, ie they take the other side of the trade. I believe Knight would have been inter…

As an example in the case of Lehman Brothers their bankruptcy didn’t affect retail customers who were protected by the SIPC and whose investment accounts were quickly move to other brokerages.

But it’s not always so sanguine. To find a bankruptcy with commingled prop and customer funds you need only look to MF Global. To find a retail brokerage bankruptcy with commingled funds you can go “Wolf of Wall Street” and look at Stratton Oakmont.

And despite how smooth everything turned out for Lehman there was a period of a day or two for some retail guys where it wasn’t exactly clear where your money was.

Re: Knightmare: A DevOps Cautionary Tale (2014)

#25

Post by brazzy »

whalesalad wrote: post #14

Back when I used to smoke I would ocassionally hang out with this guy from an investment bank that traded on the Japanese exchange. They had really cool working hours (started a lot later in the day) because we were based in Hawaii which is a few hours behind Japan. Anyway, the guy told me that they had multiple big red physical kill switches so that they could immediately turn things off if shit ever hit the fan wit…

They did have a kill switch. What they did not have was someone with the authority and guts to throw it in time.

This may have something to do with the fact that killing a HFT bot without some kind of orderly wind-down might leave you with some very expensive open positions.

Re: Knightmare: A DevOps Cautionary Tale (2014)

#26

Post by mongol »

toomuchtodo wrote: post #6

This is less DevOps and more poor software engineering practices (code reviews, unit testing, paying off your technical debt through refactoring/removing old code, etc), although properly managing and instrumenting deploys might have stemmed the bleeding and kept losses manageable. It's good though; poor decisions must have a cost. The only way to enforce good engineering practices that are human time intensive is fo…

I think at it's core it is right in the guts of DevOps. The "flag" that protects dead code is dev, and the unforeseen deployment scenario is ops. With a DevOps mindset you need to think of both. I think it is a stellar example of what can go wrong if you don't consider both the dev and the ops aspects.

Re: Knightmare: A DevOps Cautionary Tale (2014)

#27

Post by seanhunter »

manwithplan wrote: post #19
JackFr wrote: post #13

Amusing personal anecdote -- the Knight debacle caused the market in general to tumble. The week before a coworker of mine -- sure of a market drop but for other reasons -- had bought a raft or puts on the S&P 500. When I saw looking glum at work after the Knight news broke, I asked him what was wrong didn't you make a ton? Yeah, he said, but I can't get out cause my account's with Knight.

Cool story, didn't happen. There were no retail trading accounts at Knight. In fact, there was no outside money of any kind. The S&P500 fell about 0.75% on the day in question: a non-trivial decline, but not really remarkable. It was up about 0.4% on the week. Also, this is incredibly not how the OCC deals with members in default.

+1 story is definitely not true. Knight was a marketmaker and definitely not a retail broker.

Source: used to work in algo trading for GS, so this was my job, also had friends at knight during this debacle.x

Re: Knightmare: A DevOps Cautionary Tale (2014)

#28

Post by seanhunter »

alasdair_ wrote: post #20
manwithplan wrote: post #19

Earlier quoted context omitted.

Cool story, didn't happen. There were no retail trading accounts at Knight. In fact, there was no outside money of any kind. The S&P500 fell about 0.75% on the day in question: a non-trivial decline, but not really remarkable. It was up about 0.4% on the week. Also, this is incredibly not how the OCC deals with members in default.

>There were no retail trading accounts at Knight. The article states “The NYSE was planning to launch a new Retail Liquidity Program (a program meant to provide improved pricing to retail investors through retail brokers, like Knight)” This pretty strongly implies Knight was a retail broker. I assume I’m missing something- can you clarify?

You're misreading because the sentence is unclear possibly because the author doesn't know much about market structure.

The NYSE program was to provide improved pricing to retail investors through retail brokers. The people who provide the pricing to the retail brokers were market makers like Knight, Jump etc. You didn't have a brokerage account with Knight, your broker would trade in the market and Knight would take the other side.

Post reply on HN