Live data from Hacker News

CrowdStrike Update: Windows Bluescreen and Boot Loops

old.reddit.com

711–720 of 1001 posts

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#711
post #638

Isn't Crowdstrike the same company the heavily lobbied to get make all their features a requirement for government computers? https://www.opensecrets.org/federal-lobbying/clients/summary... They have plenty of money for congress, but it seem little for any kind of reasonable software development practices. This isn't the first time crowdstrike has pushed system breaking changes.

This demonstrated that Crowdstrike lacks the most basic of tests and staging environments.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#712

Perhaps a dumb question for someone who actually knows how Microsoft stuff works... Why would an anti-malware program be allowed to install a driver automatically ... or ever for that matter? Added: OK, from another post I now know Crowdstrike has some sort of kernel mode that allows this sort of catastrophe on Linux. So I guess there is a bigger question here...

> Why would an anti-malware program be allowed to install a driver automatically ... or ever for that matter?

Because malware that gets into a system will do just that -- install its own backdoor drivers -- and will then erect defense to protect itself from future updates or security actions. e.g. change the path that Windows Updater uses to download new updates, etc.

Having a kernel module that answers to CloudStrike makes it harder for that to happen, since CS has their own (non-malicious) backdoor to confirm that the rest of the stack is behaving as expected. And it's at the kernel level, so it has visibility into deeper processes that a user-space program might not (or that is easy to spoof).

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#713

I've picked the perfect day to return from vacation. Being greeted by thousands of users being mad at you and people asking for your head on a plate makes me reconsider my career choice. Here's to 12 hours of task force meetings...

Your head belongs on the plate for not being able to point back to your recommendation for failover posture enhancement such as identifying core business systems, core function roles, having fully offline emergency systems, warning of the dangers of making cloud services your only services, and then pointing to the proposed costs to implement these systems being lower than the damages caused by outage to core business services.

Move to a new career if you feel you don't have the ability to push right back against this.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#714
post #708

is there an ELI5 on how can this happen? Like i get its a boot loop, but what did crowdstrike do that cause it? How can non malicious code trigger boot loop?

Kernel driver bug that essentially defaults, then on reboot loads the same driver early on segfaults and reboots again, ad nauseum.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#715

Due to the scale I think it’s reasonable to state that in all likelihood many people have died because of this. Sure it might be hard to attribute single cases but statistically I would expect to see a general increase in probability. I used to work at MS and didn’t like their 2:1 test to dev ratio or their 0:1 ratio either and wish they spent more work on verification and improved processes instead of relying on tes…

At one company I used to work for, we had boring, airgapped systems that just worked all the time, until one day security team demanded that we must install this endpoint security software. Usually, they would fight tooth and nail to prevent devs from giving any in-house program any network access, but they didn't even blink once to give internet access to those airgapped systems because CrowdStrike agents need to talk to their mothership in AWS. It's all good, it's for better security!

It never caught any legit threat, but constantly flagged our own code. Our devs talked to security every other week to explain why this new line of code is not a threat. It generated a lot of work and security team's headcount just exploded. The software checked a lot of security checkboxes, and our CISO can sleep better at night, so I guess end of day it's all worth it.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#717
post #646

I'm confused as to how this issue is so widespread in the first place. I'm unfamiliar with how Crowdstrike works, do organizations really have no control over when these updates occur? Why can't these airlines just apply the updates in dev first? Is it the organizations fault or does Crowdstrike just deliver updates like this and there's no control? If that's just how they do it, how do they get away with this?

It's a Mossad/CIA sponsored spyware agent.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#718
post #678

Anything that has root/kernel access is a risk. It always has been. When will we learn. Probably never. Because money runs this world. So sad. Time to open a bakery and move on from this world.

Considering what Crowdstrike is intended to do, it's not really possible for it to work without running at the kernel level.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#719

Some Canonical guy I think many years ago mentioned this as their sales strategy a few year ago after a particularly nasty Windows outage: We don't ask customers to switch all systems from Windows to Ubuntu, but to consider moving maybe a third to Ubuntu so they won't sit completely helpless next time Windows fail spectacularly. While I see more and more Ubuntu systems, and recently have even spotted Landscape in the…

Sure but if that Canonical sales person was successful in that, I'd almost guarantee that after they switched the first third they'd be in there arguing to switch out the rest.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#720

The thing that amazes me is how they've rolled out such a buggy change at such a scale. I would assume that for such critical systems, there would be a gradual rollout policy, so that not everything goes down at once.

No concept of "canarying", eh?
Post reply on HN