Successes are management's, failures are individual's :)
True. But when that happens, that's a textbook sign of lack of leadership.
NYSE Tuesday opening mayhem traced to a staffer who left a backup system running
51–60 of 233 posts
Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running
#52While we're on the subject, if your company uses technology in an capacity, read the book The Phoenix Project.
https://www.amazon.com/Phoenix-Project-DevOps-Helping-Busine...
Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running
#53Earlier quoted context omitted.
This is now my headcanon explanation for why starships in Star Trek still require large crews (or crews in general).
At least in ST:VOY, it has been shown that a single hologram is capable of running the ship - although one might argue "exceptional circumstances" ;)
In Star Trek III they jury-rig the Enterprise to fly with a crew of 5, instead of the regular crew of 400.
Though in that state it can't do much more than fly: combat capabilities are strongly diminished, maintenance doesn't happen, post-combat repairs are out of the question, science missions would be much harder. On occasion the Enterprise has transported 150 passengers, so I imagine there's a lot of kitchen staff, security, etc. You only need 5 people to fly the ship, maybe 40 to fly sustainably with maintenance, but to actually accomplish their reglar mission you need the other 300 people.
Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running
#54Remember the flash crash of 2015? They let those trades actually STAND. Including options. This week’s open was nothing in comparison.
That wasn't an exchange "malfunction" in the sense that the exchange did not do what it was supposed to, was it?
Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running
#55Earlier quoted context omitted.
This. If an individual's mistake can take out your business you have a process control problem and that is owned by management.
The days of management taking responsibility for anything are over. See: not a single CEO stepping down for over hiring.
There's plenty of companies replacing their CEOs. Just today Toyota announced theirs.
Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running
#56These "issue traced to staffer" stories sound like management cover up for management/system shortcomings to me. Systems with such significant potential impact, and in industries where lack of financial investment in their continuity is a deliberate choice have very little excuse to be passing the buck to grunts for basic process flaws that can be triggered by individual error.
This. If an individual's mistake can take out your business you have a process control problem and that is owned by management.
If you have processes where there is nothing an employee can do to affect the outcome of the company you successfully built a legacy bureaucracy that is waiting to be disrupted.
Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running
#57Earlier quoted context omitted.
Some submarine operations are deliberately not automated, because if eg a sensor is broken or miscalibrated it could sink the entire boat if a computer very rapidly acts on the wrong information. Rather they do those operations with one person operating the valve/machine/device/etc, another watching and confirming readings, and the whole thing is on a constant audio link with a third person in an engineering room who…
> It's clearly not optimized for efficient use of personnel I disagree. When the sub sinks, all these people die. That's far more inefficient use of personnel. It feelf like you're putting more emphasis on the material cost ("the cost ... can be very large indeed") than on what actually matters.
It feels like a wartime-vs-peacetime priority problem.
In wartime you would care about efficiency, build the largest number of subs staffed with the minimum crew. And since training crew quickly becomes the bottleneck you would probably go for the highest degree of automation that doesn't impact production times too much. In peacetime, efficiency isn't as important. What is important is the bad PR of losing one of your submarines in a training excercise or on patrol, so crew safety becomes a much bigger concern.
Losing those sailors in war would have been a noble sacrifice for the cause, losing them to the exact same accident in peacetime is a national tragedy.
Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running
#58Is it normal to simply accept the word of an anonymous source for something so important? I genuinely don't know, anymore, but it doesn't seem like a good idea. I'd rather wait for a more thorough investigation. Especially when the story from these sources boils down to "Kevin was in charge of booting the NYSE App that morning, but he was late for work. He had a good excuse, though, he flaked! We'll have the chap straight up for lunch, no question".
Edit: I also note that this piece is lacking the traditional "The NYSE did not respond to a request for comment".
Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running
#59there is no way a person, a single person, an unauthorized person, can have access into such system/functionality like this. utter BS.
my 2c:
In reality what probably happened is previous market day and post-trading data encountered some kind of error, which triggered a cascade of problems overnight that they were unable to properly rectify. This caused delays up until market open. They were unable to fully resolve the issue, and forced with either delaying opening the market (which is a HUGE no-no) or opening with wrong data as is, they chose wrong data.
All in all a lot of people didn't get much sleep Monday. More than likely they implemented some changes or updates over the weekend that were not properly done, or they encountered some errors, and didn't have adequate controls/time to roll-back Monday night. They made the right calls too late and there was a controls process up the chain that seriously fucked up. These are the kinds of problems that get the CEO woken up in the middle of the night.
Re: NYSE Tuesday opening mayhem traced to a staffer who left a backup system running
#60Earlier quoted context omitted.
More automation -> more code -> more things to go wrong
>More automation -> more code -> more things to go wrong More people -> more entropy -> much more things to go wrong
One person might be inattentive or drunk, but it's less likely that two people are. So you institue a two-person rule. And if that's not good enough, add a third person to double-check. Maybe a supervisor to observe the people doing all of the above, to catch any mistakes or negligent behavior. Also have them write down the steps they have taken, and have somebody else read through that to verify. Just keep adding people until you are satisfied with your odds (and hope you are not making it worse through second-order effects)