Live data from Hacker News

The $440M software error at Knight Capital (2019)

henricodolfing.com

41–50 of 177 posts

Re: The $440M software error at Knight Capital (2019)

#41
I read this

>Under stock exchange rules, Knight would have been required to pay for those shares three days later. However, there was no way it could pay, since the trades were unintentional and had no source of funds behind them. The only alternatives were to try to have the trades canceled, or to sell the newly acquired shares the same day.

And then I understand why /r/WallStreeBets and /r/Antiwork is gaining traction.

All it takes is a bit of organisation and the adoption of Govt tactics and practices which is ultimately violence and then just maybe you might see a Govt that works for the people and not the criminals, but I cant picture Bernie Sanders wielding a pitchfork!

Still I see Musk was market making with his tweet. I dont think you can be any more blatant! LOL https://twitter.com/elonmusk/status/1520650036865949696?cxt=...

Re: The $440M software error at Knight Capital (2019)

#42
post #32

Earlier quoted context omitted.

Because those transactions cause other transactions, which cause others, and so on and so forth. You'd have to reset the market for the day. Imagine how pissed you'd be if you made money off Knight's mistake and it all just disappeared the next day.

Except, well, cancelling transactions obviously does happen, sometimes: https://www.reuters.com/business/lme-suspends-nickel-trading... Knight was probably too messy to rollback cleanly, but that just means it's a matter of cost/complexity/politics... if you're a big enough player, then the exchange will do you favors, like in the LME case. Free markets, lol

From the OP

  Rules were established after the “flash crash” of May 2010 to govern when trades should be canceled. Knight’s buying binge did not drive up the price of the purchased stocks by more than 30 percent, the cancellation threshold, except for six stocks. Those transactions were reversed. In the other cases, the trades stood.

Re: The $440M software error at Knight Capital (2019)

#43
post #11

Earlier quoted context omitted.

I worked in algo trading for years, eventually got out because quite frankly the level of risk I was carrying on my shoulders everyday for what I was being paid were just way out of whack, I at least personally never got the huge pay days that people talked about until after I left finance for more pure tech. Interestingly, I worked at Knight and my team pioneered trying to blow up the firm, but that was in 2004, and…

I appreciate your comment about pay. Recruiters will often tell me "it's finance so of course the pay will be substantial." Then when we get to talking numbers they're like "300k a year". Oh, you mean the going rate at a FAANG? And I have to move to New York or Chicago, work more hours, and actively work for people who I know are taking home paychecks with 7+ zeroes on them? Come on. Sometimes it's 400 plus bonus or…

I cannot confirm this. ~300k is pay (excluding sign-on) fresh out of college at a big HFT -- sufficiently senior devs make 7 figures.

Re: The $440M software error at Knight Capital (2019)

#44

Earlier quoted context omitted.

I appreciate your comment about pay. Recruiters will often tell me "it's finance so of course the pay will be substantial." Then when we get to talking numbers they're like "300k a year". Oh, you mean the going rate at a FAANG? And I have to move to New York or Chicago, work more hours, and actively work for people who I know are taking home paychecks with 7+ zeroes on them? Come on. Sometimes it's 400 plus bonus or…

The attitude that finance pays more is a leftover from a previous era. 10-15 years ago it was true: the profits from HFT were so also way, way bigger and split up amongst a much smaller group of firms. Now those firms are all in a completely competitive industry squeezing each other for basis points. Meanwhile the definition of a FAANG is that it has an effective monopoly, and these companies are taking in way more m…

> but we can’t really drop N from the acronym without a replacement

Huh, yeah. That would be quite a GAAF. Gotta come up with something before Netflix is forced out of the FAANG club.

Re: The $440M software error at Knight Capital (2019)

#45
post #11

Earlier quoted context omitted.

I worked in algo trading for years, eventually got out because quite frankly the level of risk I was carrying on my shoulders everyday for what I was being paid were just way out of whack, I at least personally never got the huge pay days that people talked about until after I left finance for more pure tech. Interestingly, I worked at Knight and my team pioneered trying to blow up the firm, but that was in 2004, and…

I appreciate your comment about pay. Recruiters will often tell me "it's finance so of course the pay will be substantial." Then when we get to talking numbers they're like "300k a year". Oh, you mean the going rate at a FAANG? And I have to move to New York or Chicago, work more hours, and actively work for people who I know are taking home paychecks with 7+ zeroes on them? Come on. Sometimes it's 400 plus bonus or…

Yeah, pay at the big banks is shit really, especially when you consider the utter lack of work/life balance. I left in 2013 making 150k, which was supposed to be supplemented by a ~40% bonus for the level I was at, but each year was "well its been a tough year..." and after getting a token amount one year, and then zeroes the next 2, after working 50-60 hour weeks, I was like I am not only done with this place, but this industry, and left for a 50% pay raise, my TC is now 4x where it was in those days. A neighbor of mine is more or less sitting in my exact seat there, and is somewhere in the 200-250k range.

That said, I went back to finance to work at one of the premier hedge funds out there, and they actually lived up to their expectations in terms of comp, that place was more like a tech firm though than any other firm I have ever worked at aside for maybe Knight. 8% annual increases were normal there. You can look in my post history back to 2018 if you want the name, I recently left after 5 years there and just want to stay out of their crosshairs- they monitor social media aggressively and there is deferred comp at stake.

At big banks, there are really only a very small number of people who are in tech that are getting paid- you have to know which questions to ask- where is the bonus pool coming from- are you "in the business" or the tech pool, which is a second class of citizen. I would have to be in a pretty bad place to ever consider going back to a bank, it was borderline abusive... always dangling the prospect of that big check that would make it worth it

Re: The $440M software error at Knight Capital (2019)

#46
Random, but I interviewed at Knight Capital for a software engineering position a few weeks before this all went down. I was in London, so the interview was done over the phone. Picture me in the evening, handwriting C to solve some problem (the fog of time too thick to remember what that problem was), then reading out what I'd written, semicolons and all, to the interviewer. Because of course there was no shared doc. I did very badly. But then, so did they.

Re: The $440M software error at Knight Capital (2019)

#47
post #43

Earlier quoted context omitted.

I appreciate your comment about pay. Recruiters will often tell me "it's finance so of course the pay will be substantial." Then when we get to talking numbers they're like "300k a year". Oh, you mean the going rate at a FAANG? And I have to move to New York or Chicago, work more hours, and actively work for people who I know are taking home paychecks with 7+ zeroes on them? Come on. Sometimes it's 400 plus bonus or…

I cannot confirm this. ~300k is pay (excluding sign-on) fresh out of college at a big HFT -- sufficiently senior devs make 7 figures.

At our firm, the numbers are similar: $600k TC for new grads ($200k base, $100k minimum first year bonus, $300k signing bonus)

Re: The $440M software error at Knight Capital (2019)

#48
post #11

Earlier quoted context omitted.

I worked in algo trading for years, eventually got out because quite frankly the level of risk I was carrying on my shoulders everyday for what I was being paid were just way out of whack, I at least personally never got the huge pay days that people talked about until after I left finance for more pure tech. Interestingly, I worked at Knight and my team pioneered trying to blow up the firm, but that was in 2004, and…

I appreciate your comment about pay. Recruiters will often tell me "it's finance so of course the pay will be substantial." Then when we get to talking numbers they're like "300k a year". Oh, you mean the going rate at a FAANG? And I have to move to New York or Chicago, work more hours, and actively work for people who I know are taking home paychecks with 7+ zeroes on them? Come on. Sometimes it's 400 plus bonus or…

Why pay more than market rate of a replaceable ML person?

Is there any realistic path for a demonstrably smart and hardworking person into that 7+zeros club? Evidence suggests no: leetcode grinders and FAANGers are not in that club, and most of them will never even make it into the 6+zeros club. Net wealth -- sure, but not income.

Re: The $440M software error at Knight Capital (2019)

#49
post #43

Earlier quoted context omitted.

I cannot confirm this. ~300k is pay (excluding sign-on) fresh out of college at a big HFT -- sufficiently senior devs make 7 figures.

At our firm, the numbers are similar: $600k TC for new grads ($200k base, $100k minimum first year bonus, $300k signing bonus)

WTF! I am at an HFT firm in Chicago, this is insane. This seems to be a lot like an offer from Radix, or Headland, or maybe Algo Dev at HRT.

Re: The $440M software error at Knight Capital (2019)

#50
post #2

The OP details how poor software engineering practices brought down a 1.4B market marker with 1400 employees in 2012. Some of the issues mentioned include: - Keeping synthetic test data generation as part of a production build. - Keeping dead code for years. - Re-purposing a feature flag. - Refactoring without regression tests. - Manual deployments without peer reviews. They forgot to update one of their servers with…

I used to work in HFT. I have seen highly variable practices in this case, including a "mini-knight" incident in the single-digit millions due to tech debt and poor test coverage. However, the most useful change that has resulted from the KCG debacle was adding several layers of kill switches, a dedicated ops team to watch trading and flip the kill switches, and embracing devops automation. There is a much more serio…

I’m curious about the “repurposed flags” part.

I wouldn’t think of flags as expensive / effortful to make more of, but clearly they must be if people are tempted to reuse them. Can you help me understand what is meant by a flag in this context, and why it would be repurposed?

Post reply on HN