Live data from Hacker News

CrowdStrike Update: Windows Bluescreen and Boot Loops

old.reddit.com

991–1000 of 1001 posts

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#992
post #538

Crowdstrike did this to our production linux fleet back on April 19th, and I've been dying to rant about it. The short version was: we're a civic tech lab, so we have a bunch of different production websites made at different times on different infrastructure. We run Crowdstrike provided by our enterprise. Crowdstrike pushed an update on a Friday evening that was incompatible with up-to-date Debian stable. So we patc…

So in a nutshell it is about corporations pushing for legislation which compels usage of their questionable products, because such products enable management to claim compliance when things go wrong, even when the things that go wrong is are the compliance ensuring products.

Sounds like a good gig to me.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#993
If I were a cloud vendor, I would provide a "CrowdStrike recovery" button which queues the recovery image and restores the system for the entire project. Why didn't hetzner, linode, DO, gcp, aws do something like this? Why leave people to their devices? Isn't this a basic application of centralization? It feels to me like this should be easier than managing your data center.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#994
post #538

Crowdstrike did this to our production linux fleet back on April 19th, and I've been dying to rant about it. The short version was: we're a civic tech lab, so we have a bunch of different production websites made at different times on different infrastructure. We run Crowdstrike provided by our enterprise. Crowdstrike pushed an update on a Friday evening that was incompatible with up-to-date Debian stable. So we patc…

This seems to be misinformation? The CrowdStrike KB says this was due to a Linux kernel bug.

---

Linux Sensor operating in user mode will be blocked from loading on specific 6.x kernel versions

Published Date: Apr 11, 2024 Symptoms

In order to not trigger a kernel bug, the Linux Sensor operating in user mode will be prevented from loading on specific 6.x kernel versions with 7.11 and later sensor versions.

Applies To Linux sensor 7.11 in user mode will be prevented from loading:

For Ubuntu/Debian kernel versions:

6.5 or 6.6

For all distributions except Ubuntu/Debian, kernel versions:

6.5 to 6.5.12

6.6 to 6.6.2

Linux sensor 7.13 in user mode will be prevented from loading:

For Ubuntu 22.04 running the 6.5 kernel:

6.5.0 - 6.5.0-1014-aws 6.5.0- 6.5.0-1015-azure 6.5.0 - 6.5.0-1014-gcp 6.5.0 - 6.5.0-24-generic 6.5.0 - 6.5.0-1015-oem Ubuntu kernel version:

6.6 to 6.6.2 For Debian kernel version:

6.5 to 6.5.12

6.6 to 6.6.2

For all distributions except Ubuntu/Debian, kernel versions:

6.5 to 6.5.12

6.6 to 6.6.2

Linux Sensors running in kernel mode are not affected.

Resolution CrowdStrike Engineering identified a bug in the Linux kernel BPF verifier, resulting in unexpected operation or instability of the Linux environment.

In detail, as part of its tasks, the verifier backtracks BPF instructions from subprograms to each program loaded by a user-space application, like the sensor. In the bugged kernel versions, this mechanism could lead to an out-of-bounds array access in the verifier code, causing a kernel oops.

This issue affects a specific range of Linux kernel versions, that CrowdStrike Engineering identified through detailed analysis of the kernel commits log. It is possible for this issue to affect other kernels if the distribution vendor chooses to utilize the problem commit.

The commit where the kernel bug was introduced is seen at https://github.com/torvalds/linux/commit/fde2a3882bd07876c14... and the commit that resolves the issue is seen at https://github.com/torvalds/linux/commit/4bb7ea946a370707315...

To avoid triggering a bug within the Linux kernel, the sensor is intentionally prevented from running in user mode for the specific distributions and kernel versions shown in the above section

These kernel versions are intentionally blocked to avoid triggering a bug within the Linux kernel. It is not a bug with the Falcon sensor. Sensors running in kernel mode are not affected.

No action required, the sensor will not load into user mode for affected kernel versions and will stay on kernel mode.

For Ubuntu 22.04 the following 6.5 kernels will load in user mode with Falcon Linux Sensor 7.13 and higher:

6.5.0-1015-aws and later 6.5.0-1016-azure and later 6.5.0-1015-gcp and later 6.5.0-25-generic and later 6.5.0-1016-oem and later

If for some reason the sensor needs to be switched back to kernel mode: Switch the Linux sensor backend to kernel mode sudo /opt/CrowdStrike/falconctl -s --backend=kernel

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#995
post #538

Crowdstrike did this to our production linux fleet back on April 19th, and I've been dying to rant about it. The short version was: we're a civic tech lab, so we have a bunch of different production websites made at different times on different infrastructure. We run Crowdstrike provided by our enterprise. Crowdstrike pushed an update on a Friday evening that was incompatible with up-to-date Debian stable. So we patc…

[deleted]

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#996

Read on Mastodon: https://infosec.exchange/@littlealex/112813425122476301 The CEO of Crowdstrike, George Kurtz, was the CTO of McAfee back in 2010 when it sent out a bad update and caused similar issues worldwide. If at first you don't succeed, .... ;-) j/k

I didn’t understand why in 2010, it didn’t seem to make most news…

Took out the entire company where I worked.

People thought it was a worm/virus — few minutes after plugging in laptop, McAfee got the DAT update, quarantined the file; which caused Windows to start countdown+reboot (leading to endless BSODs).

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#999

Throwaway account... CrowdStrike in this context is a NT kernel loadable module (a .sys file) which does syscall level interception and logs then to a separate process on the machine. It can also STOP syscalls from working if they are trying to connect out to other nodes and accessing files they shouldn't be (using some drunk ass heuristics). What happened here was they pushed a new kernel driver out to every client…

Hello:

I'm a reporter with Bloomberg News covering cybersecurity. I'm trying to learn more about this Crowdstrike update potentially bypassing staging rules and would love to hear about your experience. Would you be open to a coversation?

I'm reachable by email at jbleiberg2@bloomberg.net or on Signal at JakeBleiberg.24. Here's my Bloomberg author page: https://www.bloomberg.com/authors/AWuCZUVX-Pc/jake-bleiberg.

Thank you.

Jake

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#1000
post #538

Crowdstrike did this to our production linux fleet back on April 19th, and I've been dying to rant about it. The short version was: we're a civic tech lab, so we have a bunch of different production websites made at different times on different infrastructure. We run Crowdstrike provided by our enterprise. Crowdstrike pushed an update on a Friday evening that was incompatible with up-to-date Debian stable. So we patc…

[deleted]
Post reply on HN