Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NYSE Tuesday opening mayhem traced to a staffer who left a backup system running

bloomberg.com

Post reply on HN
Print view View original

121–130 of 233 posts

Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running

#121

Post by 1980phipsi »

bigpeopleareold wrote: post #70
mellosouls wrote: post #22

These "issue traced to staffer" stories sound like management cover up for management/system shortcomings to me. Systems with such significant potential impact, and in industries where lack of financial investment in their continuity is a deliberate choice have very little excuse to be passing the buck to grunts for basic process flaws that can be triggered by individual error.

I had a project that was really important once. I made a tiny mistake that had a big consequence - a couple of hours of potential lost revenues from our customers. I fixed my mistake with both my boss and CEO nearby. I said after I pushed the fix that I really need more resources around it. That little light of "yeah, this is important" that should have flickered didn't. :) I will not be surprised if nothing gets fix…

If you were to ask what is the probability that this specific error happens again, I would think it would be pretty low. Probably lower than a week ago. If you were to ask what is the probability that some significant, costly error happens again, I don't think the probability is that much lower than a week ago.

Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running

#122

Post by willcipriano »

PragmaticPulp wrote: post #118
mellosouls wrote: post #22

These "issue traced to staffer" stories sound like management cover up for management/system shortcomings to me. Systems with such significant potential impact, and in industries where lack of financial investment in their continuity is a deliberate choice have very little excuse to be passing the buck to grunts for basic process flaws that can be triggered by individual error.

> These "issue traced to staffer" stories sound like management cover up for management/system shortcomings to me. At the end of the day, the engineers are responsible for the engineering. Managers are responsible for managing. Shifting all responsibility for execution issues on to management can give warm fuzzies, but in reality managers aren’t all powerful in shaping execution by engineers. Companies that put all b…

Do engineers have authority over engineering? Can they overrule management on engineering issues? Whoever takes the authority gets the blame.

Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running

#123

Post by Octoth0rpe »

hgsgm wrote: post #99
Octoth0rpe wrote: post #79

Earlier quoted context omitted.

The CEO of Toyoda is becoming the chairman of their board, that doesn't feel like a CEO being replaced as punishment for poor performance in the way that people are talking about in this thread. But even when CEOs are fully ousted over issues, the golden parachute makes it barely feel like a punishment anyway. I'm having trouble thinking of a case where a CEO actually seemed to be significantly financially impacted b…

Are the golden parachutes bigger (as % of annual comp) than employee severance packages?

Do you think that honestly matters in the 10s of millions of dollars range? I certainly don't. The problematic parachutes in question are beyond enough for an excessively wealthy standard of living for the rest of their natural lives, even if it's proportionally smaller. Whether or not CEO comp should be as high as it currently is is another question entirely.

Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running

#124

Post by johnbcoughlin »

khold_stare wrote: post #104
jrochkind1 wrote: post #87

> That misled the exchange’s computers to treat the 9:30 a.m. opening bell as a continuation of trading, and so they skipped the day’s opening auctions that neatly set initial prices. I didn't even know about this process. I don't know much about trading, but it surprises me that there is a separate process for setting prices at the start of trading, and that if it's missed, chaotic prices result. Is this related to…

Worked in HFT for a few years. The reason why most markets are not open 24 hours is more human, and just historical - aligned with people's 9-5 workday. There are also pre open and post close sessions of trading but it's much less liquid. Futures markets are open almost 24 hours. Even there, it's down for some time daily. Personally I think it's actually inertia that keeps existing markets this way - the systems of t…

Matt Levine suggested that the chaos after opening was mainly due to market orders executing at ridiculous prices. Like, a limit buy for half the "real price" is the first buy order to get in the door, and that gets matched with a market sell order.

Does that track with your understanding?

Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running

#125

Post by DrBazza »

zaroth wrote: post #107
DrBazza wrote: post #105

Earlier quoted context omitted.

I'll make an extreme comparison: "Kill one man, and you are a murderer. Kill millions of men, and you are a conqueror" If you make some idiotic financial decision near the bottom of the management tree, such as... over hiring, you'll likely lose your job or get demoted. Do it as a CEO, and get a huge bonus. [1] https://en.wikipedia.org/wiki/Jean_Rostand

But it’s absurd. Companies are not supposed to only ever hire . Some things are cyclical and you need more people for some amount of time, and then you find you need less. It’s not always predictable/seasonal like farming or holiday rush. Is it wrong for a company to respond to market effects? That there was a layoff isn’t necessarily a sign a company did anything wrong… I think how they actually do the layoff certai…

It’s not hiring though. It’s overhiring.

I’ve forgotten which FAANG it is. But one of them still has more employees than last year even after layoffs. It’s offensive.

Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running

#126

Post by ngz00 »

ynniv wrote: post #106

It's easy to throw shade when Bloomberg writes an article that puts the blame on "a staffer". Having worked near some of these systems, the engineering and process are actually quite good. How many companies publish their private network topology, service p99.9 in microseconds, and detailed pricing on the open web? They're in a painfully competitive global market that's ambivalent to names on buildings. In a week or…

I worked there and I can say that this is not accurate at all. It is very much a blame culture. I've seen people fired for less severe incidents. Beyond the core technology of the Pillar engine, the place is not comparable to a modern tech company in almost any way.

Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running

#127

Post by anonred »

ynniv wrote: post #106

It's easy to throw shade when Bloomberg writes an article that puts the blame on "a staffer". Having worked near some of these systems, the engineering and process are actually quite good. How many companies publish their private network topology, service p99.9 in microseconds, and detailed pricing on the open web? They're in a painfully competitive global market that's ambivalent to names on buildings. In a week or…

Serious question: If someone is smart and capable enough to work on tangible things like AI systems or self-driving cars, why should they choose the NYSE outside of pure monetary reasons or affinity for a “modern” tech stack?

Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running

#128

Post by throwaway122095 »

ngz00 wrote: post #126
ynniv wrote: post #106

It's easy to throw shade when Bloomberg writes an article that puts the blame on "a staffer". Having worked near some of these systems, the engineering and process are actually quite good. How many companies publish their private network topology, service p99.9 in microseconds, and detailed pricing on the open web? They're in a painfully competitive global market that's ambivalent to names on buildings. In a week or…

I worked there and I can say that this is not accurate at all. It is very much a blame culture. I've seen people fired for less severe incidents. Beyond the core technology of the Pillar engine, the place is not comparable to a modern tech company in almost any way.

As somebody who worked with them as a client, I can confirm this. There is currently a spec-level bug with their core Pillar engine and it was essentially bounced between several different teams and ultimately ignored as nobody's problem.

Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running

#129

Post by jacquesm »

ynniv wrote: post #106

It's easy to throw shade when Bloomberg writes an article that puts the blame on "a staffer". Having worked near some of these systems, the engineering and process are actually quite good. How many companies publish their private network topology, service p99.9 in microseconds, and detailed pricing on the open web? They're in a painfully competitive global market that's ambivalent to names on buildings. In a week or…

Just naming a 'staffer' though seems to already be a way to apportion blame to a segment of the employees, insulating management from what was done. Named or not doesn't really matter, clearly blame is being assigned.

Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running

#130

Post by horsawlarway »

itsoktocry wrote: post #90
thisarticle wrote: post #32

Earlier quoted context omitted.

The days of management taking responsibility for anything are over. See: not a single CEO stepping down for over hiring.

> See: not a single CEO stepping down for over hiring. Wait, what? You think a CEO should step down because their management over-hired a relatively small proportion of employees and had to do some layoffs?

Not really sure why you're getting downvoted, other than to assume an emotional reaction from the community to layoffs impacting tech.

Frankly - People seem to be forgetting that until 2013, MS was still doing stack ranking and routinely letting go of the bottom 10% of their workforce (and they were hardly the only ones doing it...)

I don't see it as unusual AT ALL that these companies are doing a wave of cuts to headcounts after the large hiring sprees during covid. Especially as interest rates rise, so they're looking to lower debt burdens in the short term and pay off loans made at low interest rates instead of rolling into a higher interest loan in the new environment.

If anything... I'd expect the exact opposite - a CEO that fails to address cost centers as debt becomes more expensive is a liability, and someone the board might be looking to replace (ask to step down).

---

Does that mean I'm not sympathetic to those who've lost jobs? Of course not.

But tech had to rev the engine pretty hard to handle the extra load during covid when everyone was indoors and doing things online, and now that demand has dropped. So they're letting off the gas pedal.

If folks don't like it - blame the game. Work to unionize. Work to incentivize co-ops and shared ownership. Work to increase taxation on these companies and their highest earners (which... if you're in the tech industry almost certainly includes YOU). Don't go work for giant tech conglomerates and then act surprised when they act like giant tech conglomerates...

Post reply on HN