How to lose $172,222 a second for 45 minutes • Hacker News Board

Re: How to lose $172,222 a second for 45 minutes

#231

Post by Dylan16807 » Wed, Oct 23, 2013, 4:20 AM UTC

jrochkind1 wrote: post #230

comrade_ogilvy wrote: post #189

Earlier quoted context omitted.

Yours is the correct viewpoint: it is incompatible with human cognition. If an alert system is not perceived as highly reliable in directing positive action, then the humans involved will inevitably disable the alert system, either by pulling out a screwdriver or rewriting their mental rubrics to ignore the messages as noise. Knight Capital is just the finance version of Three Mile Island and Deepwater Horizon -- the…

in the ancestral comment here, it wasn't even an alert system! It was just "your job is to sit there all day and watch every single transaction and flag the aberrant one"!!

"Whoa this transaction is way bigger than normal. So is this one. And this one."

People ignore repetitive things, but they usually notice when it changes. They can tell you that it's shaped different or explain how it sounds different from normal.

If this system made dissimilar transactions look very similar to the monitor, then it is to blame, not the idea of having a monitor at all.

Re: How to lose $172,222 a second for 45 minutes

#232

Post by avty » Wed, Oct 23, 2013, 4:42 AM UTC

Someone made $172,222 a second for 45 minutes on the opposite side of these trades.

Re: How to lose $172,222 a second for 45 minutes

#233

Post by icu » Wed, Oct 23, 2013, 8:08 AM UTC

chollida1 wrote: post #214

adambratt wrote: post #9

The week after this we had a trader in our office who had a meeting at Knight on the morning it happened. He said he saw the whole dev team just power off and go home at 11am, followed quickly by the rest of the employees. At that point, there was nothing they could do. The craziest thing is that it went on for so long. No one caught it until their own traders so it come across Bloomberg and CNBC. They actually thoug…

I dont' want to call your friend a liar, but this is most likely false. > The week after this we had a trader in our office who had a meeting at Knight on the morning it happened. > He said he saw the whole dev team just power off and go home at 11am, followed quickly by the rest of the employees. 1) Dev and Trading/Sales happen at different physical locations. 2) I actually know someone who spent their day cleaning…

I don't know about Knight but in the big London trading desks developers and traders are in the same building for just this reason. As you can imagine traders can be quite demanding when things go wrong. Also plugs will get pulled if things are going horribly wrong.

Re: How to lose $172,222 a second for 45 minutes

#234

Post by alephnil » Wed, Oct 23, 2013, 9:32 AM UTC

lhaussknecht wrote: post #73

jpatokal wrote: post #46

Just another reminder of how systems that you'd think are rock solid often aren't. In my previous life working with telcos, I once tried to teach a particularly huge customer how to use CVS how to manage configurations across a 10+ machine cluster of machines. They didn't see any value in it, so they stuck to their good old process of SSHing into each machine individually, "cp config.xml config.xml.20131022", and the…

Reminds me of a colleague who RDPed into each of our 140 subsidiaries to change a config file. He had a list of servers on his desk and ticked off every server. Took him the whole day to apply the changes.

When the service guy in a former job quit, the databases at the customers' production servers started to be corrupted. It appeared that he had manually logged into the database at every single customer regularly and fixed the problem on site. I don't know if had told the responsible developers (I worked on a different product), but after it was apparent that the system was not stable.

Re: How to lose $172,222 a second for 45 minutes

#235

Post by mathattack » Wed, Oct 23, 2013, 10:33 AM UTC

Zigurd wrote: post #226

mathattack wrote: post #169

Earlier quoted context omitted.

Legacy telcos are the epitome of large big institutions where many of the best talents leave. After just a few years of working with them, I couldn't believe they could get a dial tone. How did they produce so much great R&D?

> How did they produce so much great R&D? By having negotiated profit margins. Bell Labs was meant to soak up excess money. What we are now nostalgic for would never have happened without an interval during which a telco was considered a "natural monopoly."

But didn't Bell Labs have the telco mentality? Or how did they protect themselves from it?

Re: How to lose $172,222 a second for 45 minutes

#236

Post by mathattack » Wed, Oct 23, 2013, 10:34 AM UTC

pnathan wrote: post #178

cstross wrote: post #100

Earlier quoted context omitted.

Two words: job security. (Also: lots of folks really don't enjoy learning new stuff. Or new ways of working. No, they really don't. Even if the new techniques are vastly better and more efficient. Put this down to a human cognitive bias favouring the tried-and-trusted over new-and-untested. There's a lot to be said for that when you're a neolithic hunter-gatherer or an iron-age peasant -- if you try something new and…

Totally true. Even when the path is seamless, initial costs are nearly zilch... learning new things and applying them is anathema to many devs.

You can get blamed for changing something! The power if inertia....

Re: How to lose $172,222 a second for 45 minutes

#237

Post by mgav » Wed, Oct 23, 2013, 11:28 AM UTC

Ricapar wrote: post #229

simoncion wrote: post #227

Earlier quoted context omitted.

Would you kindly provide the un-minified version of your link? This isn't Twitter, and HN has its own method of shortening overlong links while also displaying the original link target.

Un-minified link: http://books.google.com/books?id=geCHWBx-e9EC&pg=PA126&lpg=P...

Gracias Ricapar.

Re: How to lose $172,222 a second for 45 minutes

#238

Post by strongvigilance » Wed, Oct 23, 2013, 11:36 AM UTC

vidarh wrote: post #113

strongvigilance wrote: post #108

Earlier quoted context omitted.

Why have a limit on how close people can be to the exchange? For one thing, everyone would just colo at exactly the minimum distance, achieving nothing. For another, ability to colo is not really an issue. If you wanted to create an HFT startup, colo is not going to be a major cost compared to hiring developers.

A limit on distance is in effect a speed limit. It means anyone that is able to reach the limit has a shot at competing. Whether that is good or not, or makes a difference or not is another discussion, but you can certainly affect the competitive situation massively that way.

Competition is already high, and colocating is not that expensive for anyone with the ability to compete.

A lot of the loudest criticism of HFT is that it's too competitive - a lot of people who used to make a comfortable living from the bid/ask spread are no longer able to due to computers driving down profit margin.

Re: How to lose $172,222 a second for 45 minutes

#239

Post by KMag » Wed, Oct 23, 2013, 1:53 PM UTC

manishsharan wrote: post #33

Don't humans also make similar large scale mistakes? Merill Lynch's infamous London whale comes to mind. Also. I could be wrong but aren't most of derivatives a zero sum game: don't I have to lose money on my puts for you to make money on your calls ? Didn't so many people lose money on securities because they misunderstood their exposure ? The Knight computer error was spectacular and catastrophic but us humans have…

If you consider gains and losses just in terms of dollar value, then options are zero-sum. However, if you take non-linear utility functions into account, then a fairly priced transfer of risk from a more risk-adverse party to a less risk-adverse party is a win-win situation.

For instance, let's say a large institutional investor determines that a 5-yr 10-yr flattener on South Whereisitstan bonds is very attractively priced, but can't stomach the risk of the 10-yr yield going through the roof. They pay an investment bank to create some OTC options on the Whereisitstan 10-year notes and several medium-sized hedge funds take the other side. If the hedge funds are right, the big institution over-pays for the options in strict dollar terms, but the options allow the institution to enter into a very attractive bond trade they otherwise would have been unwilling to enter. In this case, everyone could win, even though the big institution takes a (both realized and statistical) loss on the options.

Re: How to lose $172,222 a second for 45 minutes

#240

Post by KMag » Wed, Oct 23, 2013, 2:30 PM UTC

hderms wrote: post #164

koliber wrote: post #143

Earlier quoted context omitted.

A lot of the HFT high-frequencyness comes from lots of orders beng sent to market. Only a very small number of these orders result in transactions. A transaction fee would not affect most of the order flow which never gets executed.

there should be a small tax per order put out, even if cancelled and a small percentage-based tax on each trade. If that behavior has been determined to exacerbate price declines that externality should be captured in a tax and de-incentivized properly.

If your goal is to reduce the maximum frequency of order inserts/cancels, it makes more sense to directly charge based on frequency.

Several exchanges (the Hong Kong Stock Exchange, for one) rate-limit each connection to the exchange, charging fees based on the number of transactions per second allowed on the connection.