Live data from Hacker News

CrowdStrike Update: Windows Bluescreen and Boot Loops

old.reddit.com

721–730 of 1001 posts

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#721

I work for a diesel truck repair facility and just locked up the doors after a 40 minute day :( . - lifts wont operate. - cant disarm the building alarms. (have been blaring nonstop...) - cranes are all locked in standby/return/err. - laser aligners are all offline. - lathe hardware runs but controllers are all down. - cant email suppliers. - phones are all down. - HVAC is also down for some reason (its getting hot i…

wow, why do lifts require an OS?

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#722

I work for a diesel truck repair facility and just locked up the doors after a 40 minute day :( . - lifts wont operate. - cant disarm the building alarms. (have been blaring nonstop...) - cranes are all locked in standby/return/err. - laser aligners are all offline. - lathe hardware runs but controllers are all down. - cant email suppliers. - phones are all down. - HVAC is also down for some reason (its getting hot i…

How come lifts and cranes are affected by this?

Are they somehow controlled remotely? or do they need to ping a central server to be able to operate?

I can see how alarms, email and phones are affected but the heavy machinery?

(Clearly not familiar with any of these things so I am genuinely curious)

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#723
post #708

is there an ELI5 on how can this happen? Like i get its a boot loop, but what did crowdstrike do that cause it? How can non malicious code trigger boot loop?

Crowdstrike needs to be the first person in the room so that they can act like the boss. If other people show up before crowdstrike, there's a possibility that they'll somehow prevent crowdstrike from being the boss. For this reason, crowdstrike integrates with the boot process in ways that most software doesn't.

Their ability to monitor and intervene against all software on the system also puts them in a position to break all software on the system.

more accurately: s/boss/most informed spy/g

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#724
It's not the first time they pull something similar...1 month ago: "CrowdStrike bug maxes out 100% of CPU, requires Windows reboots" - https://www.thestack.technology/crowdstrike-bug-maxes-out-10...

75 Billion dollars valuation, CNBC Analysts praising the company this morning on how well the company is run!...When in reality they can't master the most basic of the phased deployment methodologies known for 20 years...

Hundreds of handsomely paid CTO's, at companies with billions of dollars in valuations, critical healthcare, airlines, who can't master the most basic of the concepts of "Everything fails all the time"...

This whole industry is depressing....

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#725

This event is predicted in Sydney Dekker’s book “Drift into Failure”, which basically postulates that in order to prevent local failure we setup failure prevention systems that increase the complexity beyond our ability to handle, and introduce systemic failures that are global. It’s a sobering book to read if you ever thought we could make systems fault tolerant.

Also a major point in the Black Swan. In the Black Swan, Taleb describes that it is better for banks to fail more often than for them to be protected from any adversity. Eventually they will become "too big to fail". If something is too big to fail, you are fragile to a catastrophic failure.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#726
post #724

It's not the first time they pull something similar...1 month ago: "CrowdStrike bug maxes out 100% of CPU, requires Windows reboots" - https://www.thestack.technology/crowdstrike-bug-maxes-out-10... 75 Billion dollars valuation, CNBC Analysts praising the company this morning on how well the company is run!...When in reality they can't master the most basic of the phased deployment methodologies known for 20 years...…

This borked our dispatch/911 call center then as well. However, it wasn't as bad as this one. This outage put our entire public safety system into the stone age and with that we were at stone age efficiency.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#727
post #432

When you see the size if the impact across the world, the number of people who will die because hospital, emergency and logistics systems are down… You don’t need conventional war any more. State actors can just focus on targeting widely deployed “security systems” that will bring down whole economies and bring as much death and financial damage as a missile, while denying any involvement…

I was in my 20s during the peak hysteria of post-9/11 and GWOT. I had to cope with the hysteria hyped 24/7 by media and DHS of a constant terror threat to determine if it was real.

The fact that global infra is so flimsy and vulnerable brought me tremendous relief. If the terror threats were real, we would have been experiencing infrastructure attacks daily.

I remember driving through rural California thinking if the terrorist cells were everywhere, they could trivially

I've read a lot of cyber security books like Countdown to Zero Day, Sandworm, Ghost in the Wires and each one brings me relief. Many of our industrial systems have the most flimsy, pathetic , unencrypted & uncredentialed wireless control protocols that are vulnerable to remote attack.

The fact that we rarely see incidents like this, and when they do happen, they are due to gross negligence rather than malice, is a tremendous relief.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#728

How long before companies start consciously de-risking by replacing general-purpose systems like Windows with newer systems with smaller attack surfaces? Why does an airline need to use Windows at all for operations? From what I’ve seen, their backend systems are still running on mainframes. The terminals are accessed on PCs running Windows, but those could trivially be replaced with iPadOS devices that are more lock…

Embedded Windows has always seemed like an oxymoron to me.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#729
post #646

I'm confused as to how this issue is so widespread in the first place. I'm unfamiliar with how Crowdstrike works, do organizations really have no control over when these updates occur? Why can't these airlines just apply the updates in dev first? Is it the organizations fault or does Crowdstrike just deliver updates like this and there's no control? If that's just how they do it, how do they get away with this?

Presumably endpoint detection & response (EDR) agents need to do things like dynamically fetch new malware signatures at runtime, which is understandable. But you'd think that would be treated as new "content", something they're designed to handle in day-to-day operation, hence very low risk.

That's totally different to deploying new "code", i.e. new versions of the agent itself. You'd expect that to be treated as a software update like any other, so their customers can control the roll out as part of their own change management processes, with separate environments, extensive testing, staggered deployments, etc.

I wonder if such a content vs. code distinction exists? Or has EDR software gotten so complex (e.g. with malware sandboxing) that such a distinction can't easily be made any more?

In any case, vendors shouldn't be able to push out software updates that circumvent everyone's change management processes! Looking forward to the postmortem.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#730

This event is predicted in Sydney Dekker’s book “Drift into Failure”, which basically postulates that in order to prevent local failure we setup failure prevention systems that increase the complexity beyond our ability to handle, and introduce systemic failures that are global. It’s a sobering book to read if you ever thought we could make systems fault tolerant.

It's also in line with arguments made by Ted Kaczynski (the Unabomber)

> Why must everything collapse? Because, [Kaczynski] says, natural-selection-like competition only works when competing entities have scales of transport and talk that are much less than the scale of the entire system within which they compete. That is, things can work fine when bacteria who each move and talk across only meters compete across an entire planet. The failure of one bacteria doesn’t then threaten the planet. But when competing systems become complex and coupled on global scales, then there are always only a few such systems that matter, and breakdowns often have global scopes.

https://www.overcomingbias.com/p/kaczynskis-collapse-theoryh...

https://en.wikipedia.org/wiki/Anti-Tech_Revolution

Post reply on HN