Here's a visual representation of flight cancellations and delays at major US airports https://www.flightaware.com/miserymap/
this animations were all the rage in Chinese wechat while this was going on
CrowdStrike Update: Windows Bluescreen and Boot Loops
961–970 of 1001 posts
Re: CrowdStrike Update: Windows Bluescreen and Boot Loops
#962The whole thing needs to be redesigned, so that antivirus and EDR solutions do not require such high privilege. We need a high-performance way for a possibly privileged service to export all the data that is needed for a decision, and then let the AV/EDR do its thing. If the AV/EDR is broken by an update, fine. At least the system won't go down.
Re: CrowdStrike Update: Windows Bluescreen and Boot Loops
#963Microsoft are going to be pissed that this is widely being discussed as a Microsoft outage. Do AV vendors like Crowdstrike need a license or something from Microsoft to push these kernal driver based things? Or is it just like anyone can make one?
yes, and they have two. it is a windows problem.
Re: CrowdStrike Update: Windows Bluescreen and Boot Loops
#964Have we failed as an industry?
Re: CrowdStrike Update: Windows Bluescreen and Boot Loops
#965Allow me to give a different, information-theoretic, perspective. How much damage can flipping a single bit cause? How much damage can altering two bits cause?
The fanout is a robustness measure on systems. If we can control the fanout we increase reliability. If all it takes is a handful of bits in a 3rd party update to kill IT infrastructure, we are doing it wrong.
Re: CrowdStrike Update: Windows Bluescreen and Boot Loops
#966Allow me to give a different, information-theoretic, perspective. How much damage can flipping a single bit cause? How much damage can altering two bits cause? The fanout is a robustness measure on systems. If we can control the fanout we increase reliability. If all it takes is a handful of bits in a 3rd party update to kill IT infrastructure, we are doing it wrong.
Are you suggesting that a 3kb update be tested 3k times to assess the impact of each possible bit flip, and 9M times for the impact of each possible pair bit flips?
Because I think that's effort better spent in other ways.
Re: CrowdStrike Update: Windows Bluescreen and Boot Loops
#967Question : was this update delivered by Crowdstrike’s update agent or Windows Update ?
Re: CrowdStrike Update: Windows Bluescreen and Boot Loops
#968The front page of https://www.accel.com/
"Fail Fast. Evolve Faster"
Re: CrowdStrike Update: Windows Bluescreen and Boot Loops
#969> We have collaborated with Intel to remediate affected hosts remotely using Intel vPro and with Active Management Technology.
This worries me. Does this mean intel has access to remotely access my machine?!?!
Re: CrowdStrike Update: Windows Bluescreen and Boot Loops
#970This event is predicted in Sydney Dekker’s book “Drift into Failure”, which basically postulates that in order to prevent local failure we setup failure prevention systems that increase the complexity beyond our ability to handle, and introduce systemic failures that are global. It’s a sobering book to read if you ever thought we could make systems fault tolerant.
I think a surprising amount of people already share this view, even if they don't go into extensive treatment with references like Dekker presumably does (I haven't read it).
I suspect most people in power just don't subscribe to that. which is precisely why it's systemic to see the engineer shouting "no!" when John CEO says "we're doing it anyway." I'm not sure this is something you can just teach, because the audience definitely has reservations about adopting it.