51–60 of 294 posts
Actually it's a big part of the point: they have a system that works with dead code in it. If you remove that dead code perhaps it unwittingly breaks something else.
That kinds of chesterson's fence is a good practice.
Earlier quoted context omitted.
Leeching implies someone has found a way to skim value from you without providing value. Someone taking on loads of risk to carry out your commands efficiently is providing value. You can argue whether they are doing so competently or not, or whether they are pricing optimally or not, but they are not just ”takers” or “leeches”.
What risk are they taking exactly? Bugs ruining the business isn't meaningful risk for the customer. It isn't like day traders are at risk of going bankrupt due to that after all. They claim liquidity is their value but given how they act they don't seem to be providing measurable liquidity, either in terms of price or volume. (Yes they increase volume by getting in the middle of trades but that isn't useful volume..…
There's operational risk, like what brought down Knight Capital, that's a type of risk. Or the risk that you will be put out of business by competition because you were too slow to innovate while burning through all your cash runway. HFT firms face the same risks that other types of businesses face. Smaller HFT firms fail often, and larger firms tend to stay around (although sometimes they also fail and often they shrink), which is similar to many mature competitive industries.
> given how they act they don't seem to be providing measurable liquidity
I'm not sure "How they act" should inform one's perspective on the empirical question of whether or not they are adding to liquidity. There is a lot of serious debate and research that has gone into that question.
A high risk situation like this would make the idea of using LLMs for this as not an option; before someone puts out a 'use-case' for a LLM to fix this issue.
I'm sorry to preempt the thought of this in advance, but it would not.
I'm not sure how automated deployments would have solved this problem. In fact, if anything, it would have magnified the impact and fallout of the problem. Substitute "a developer forgot to upload the code to one of the servers" for "the deployment agent errored while downloading the new binary/code onto the server and a bug in the agent prevented the error from being surfaced." Now you have the same failure mode, an…
> The blame here lies squarely with the developers--the code was written in a non-backwards-compatible way. The blame completely lies with the risk management team. The market knew there was a terrible problem, Knight knew there was a problem, yet it took 45 minutes of trying various hotfixes before they ceased trading. Either because they didn't have a kill switch, or because no one was empowered to pull the kill sw…
I have always found this story fascinating; in my junior days I worked at a relatively big adtech platform (ie billions of impressions per day) and as cowboy as we were about lots of things, all our systems always had kill switches that could stop spending money and I could have pulled them with minimal red tape if I suspected something was wrong.
And this was for a platform where our max loss for an hour would have hurt but not killed the business (maybe a six figure loss), I can’t imagine not having layers of risk management systems in HFT software.
It's not the worst mistake in the story, but it's not "not the point." A proactive approach to pruning dead functionality would have resulted in a less complex, better-understood piece of software with less potential to go haywire. Driving relentlessly forward without doing this kind of maintenance work is a risk, calculated or otherwise.
I'm not sure how automated deployments would have solved this problem. In fact, if anything, it would have magnified the impact and fallout of the problem. Substitute "a developer forgot to upload the code to one of the servers" for "the deployment agent errored while downloading the new binary/code onto the server and a bug in the agent prevented the error from being surfaced." Now you have the same failure mode, an…
> The blame here lies squarely with the developers--the code was written in a non-backwards-compatible way. The blame completely lies with the risk management team. The market knew there was a terrible problem, Knight knew there was a problem, yet it took 45 minutes of trying various hotfixes before they ceased trading. Either because they didn't have a kill switch, or because no one was empowered to pull the kill sw…
Ah, Knight Capital. The warning story for every quant trader / engineer. This is what people don't realize when they say HFT (high frequency trading) is risk-free, leeching off people, etc. You make a million every day with very little volatility (the traditional way of quantifying "risk" in finance) but one little mistake, and you're gone. The technical term is "picking up pennies in front of a steamroller (train)".…
I'm a proponent of both. But generally I hold more respect for actual market makers who hold positions and can warehouse risk.
Of course there were other mistakes here etc., but the issue wouldn't have been possible if this weren't the case.
> (why code that had been dead for 8-years was still present in the code base is a mystery, but that’s not the point). Actually it's a big part of the point: they have a system that works with dead code in it. If you remove that dead code perhaps it unwittingly breaks something else. That kinds of chesterson's fence is a good practice.
My team's systems play a critical role for several $100M of sales per day, such that if our systems go down for long enough, these sales will be lost. Long enough means at least several hours and in this time frame we can get things back to a good state, often without much external impact. We too have manual processes in place, but for any manual process we document the rollback steps (before starting) and monitor th…
> My team's systems play a critical role for several $100M of sales per day, such that if our systems go down for long enough, these sales will be lost. Would they? Or would they just happen later? In a lot of cases in regular commerce, or even B2B, the same sales can often be attempted again by the client for a little later, it's not "now or never". As a user I have retried things I wanted to buy when a vendor was d…
It's also probably true that a one-day outage would have a negative net present value (taking into account all future sales) far exceeding the daily loss in sales, due to loss of customer goodwill.