All US flights are grounded too. The people I was traveling with cant check into hotels
CrowdStrike Update: Windows Bluescreen and Boot Loops
91–100 of 1001 posts
Re: CrowdStrike Update: Windows Bluescreen and Boot Loops
#92The impact of this will be profound! Obviously bugs are inevitable, but why this wasn't progressively rolled out is beyond me.
My understanding is that multiple recent versions are affected.
Re: CrowdStrike Update: Windows Bluescreen and Boot Loops
#93This is why you don't make changes on a Friday. Lots of weekends absolutely ruined now.
Early weekend here :D
Re: CrowdStrike Update: Windows Bluescreen and Boot Loops
#94We are a major CS client, with 50k windows-based endpoints or so. All down. There exists a workaround but CS does not make it clear whether this means running without protection or not. (The workaround does get the windows boxes unstuck from the boot loop, but they do appear offline in the CS host management console - which of course may have many reasons).
Yeah, the simple renaming .sys files in safe mode does seem like it would inhibit protection.
Re: CrowdStrike Update: Windows Bluescreen and Boot Loops
#95[deleted]
Re: CrowdStrike Update: Windows Bluescreen and Boot Loops
#96Maybe the world can finally reconsider their use of software products that cater to security theater. And the politics in companies which lead to things like this being introduced ("nobody gets fired for buying IBM").
Edit: took out a bit of snark.
Re: CrowdStrike Update: Windows Bluescreen and Boot Loops
#97The biggest mistake here is running a global update on a Friday. Disrespect to every sysadmin worldwide.
Disrespect to every CIO to make their business depend on a single operating system, running automatic updates of system software without any canaries and phased deployments.
Re: CrowdStrike Update: Windows Bluescreen and Boot Loops
#98In terms of analysing risk factors to minimise something like this happening again, what are the factors at play here? A Crowdstrike update being able to blue-screen Windows Desktops and Servers. Whilst Crowdstrike are going to cop a potentially existential-threatening amount of blame, an application shouldn't be able to do this kind of damage to an operating system. This makes me think that, maybe, Crowdstrike were…
maybe they installed crowdstrike because they wanted updated without testing. and crowdstrike failed at testing them in their environment.
sounds like they didnt test all cases and stumbled on a windows bug
Re: CrowdStrike Update: Windows Bluescreen and Boot Loops
#99Have spent all my afternoon and all evening on a bridge trying to support flailing systems. Was supposed to be on a plane in 5 hours to start my vacation. Guaranteed it's not gonna happen.
With hearing 911 and other safety critical systems going down, I hope that the worst that comes out of this is a couple delayed flights and a couple missed bank payments.
Re: CrowdStrike Update: Windows Bluescreen and Boot Loops
#100In terms of analysing risk factors to minimise something like this happening again, what are the factors at play here? A Crowdstrike update being able to blue-screen Windows Desktops and Servers. Whilst Crowdstrike are going to cop a potentially existential-threatening amount of blame, an application shouldn't be able to do this kind of damage to an operating system. This makes me think that, maybe, Crowdstrike were…
My experience working with Crowdstrike was that they were super arrogant about these risks. I was working on a ~50k enterprise rollout, and our CS guy was very belligerent about how long we were taking to do it, how much testing we wanted to do, the way that we were staggering roll outs and managing rollback plans. He didn’t think any of this was necessary, that we should roll it out in one fell swoop, have everything to auto-update all the time, and constantly yapped about how many bigger enterprises than ours completed their rollouts in just a couple of weeks.
He actually threatened to fire us as a client because he claimed he didn’t want the CS brand associated with an org that wasn’t “fully protected” by CS. By far the worst vendor contact I’ve ever had. I’ve had nicer meetings with Oracle lawyers than I was having with this guy. I hope this sort of thing humbles them a little.