Why the CrowdStrike bug hit banks hard • Hacker News Board

Re: Why the CrowdStrike bug hit banks hard

#11

Post by btbuildem » Wed, Jul 31, 2024, 3:38 PM UTC

deepsun wrote: post #3

I'm still amazed how the blame shifted from Microsoft to CrowdStrike. Yes, CrowdStrike update caused that -- but applications fail all the time. It was Microsoft's oversight to put it on Windows critical path. And banks/airlines etc were hit hard because their _Windows_ didn't boot, not because of an application crash on a perfectly working Windows.

Isn't corporate malware by definition on the "critical path"? The article outlines the reasons why that jank runs in kernel space, and why MS is unable to "downgrade" it to userspace.

Re: Why the CrowdStrike bug hit banks hard

#12

Post by btbuildem » Wed, Jul 31, 2024, 3:39 PM UTC

The takeaway from this article seems to be: buy crowdstrike shares, because major corps are unable to make any changes, and will continue to pay licensing fees for this "service" for the foreseeable future.

Re: Why the CrowdStrike bug hit banks hard

#13

Post by ortusdux » Wed, Jul 31, 2024, 3:39 PM UTC

deepsun wrote: post #3

I'm still amazed how the blame shifted from Microsoft to CrowdStrike. Yes, CrowdStrike update caused that -- but applications fail all the time. It was Microsoft's oversight to put it on Windows critical path. And banks/airlines etc were hit hard because their _Windows_ didn't boot, not because of an application crash on a perfectly working Windows.

Is there any merit to Microsoft's argument that the EU forced them into keeping their kernel accessible by 3rd parties?

https://www.theregister.com/2024/07/22/windows_crowdstrike_k...

Re: Why the CrowdStrike bug hit banks hard

#14

Post by Retr0id » Wed, Jul 31, 2024, 3:41 PM UTC

> For historical reasons, that area where almost everything executes is called “userspace.”

It's an old term at this point, but I don't think the reasons for it being called "userspace" have changed or become outdated since then, so I wouldn't call them historic per se.

Re: Why the CrowdStrike bug hit banks hard

#15

Post by [deleted] » Wed, Jul 31, 2024, 3:41 PM UTC

deepsun wrote: post #3

I'm still amazed how the blame shifted from Microsoft to CrowdStrike. Yes, CrowdStrike update caused that -- but applications fail all the time. It was Microsoft's oversight to put it on Windows critical path. And banks/airlines etc were hit hard because their _Windows_ didn't boot, not because of an application crash on a perfectly working Windows.

[deleted]

Re: Why the CrowdStrike bug hit banks hard

#16

Post by shombaboor » Wed, Jul 31, 2024, 3:41 PM UTC

klodolph wrote: post #6

deepsun wrote: post #3

I'm still amazed how the blame shifted from Microsoft to CrowdStrike. Yes, CrowdStrike update caused that -- but applications fail all the time. It was Microsoft's oversight to put it on Windows critical path. And banks/airlines etc were hit hard because their _Windows_ didn't boot, not because of an application crash on a perfectly working Windows.

How did Microsoft put it on the Windows critical path? (Informational question—I’m not following the issue super closely, but I thought CrowdStrike was a third-party system. Crowdstrike was wrong to put so much code in the kernel. Microsoft was reportedly legally bound to provide this access and allow third-party code to run in the kernel.)

There was an interesting article that these third parties who lobbied to run in the kernel and microsoft acquiesced about 20 years ago which led us down this path. https://web.archive.org/web/20061023112233/http://software.s...

Re: Why the CrowdStrike bug hit banks hard

#17

Post by fmbb » Wed, Jul 31, 2024, 3:42 PM UTC

Cthulhu_ wrote: post #7

deepsun wrote: post #3

I'm still amazed how the blame shifted from Microsoft to CrowdStrike. Yes, CrowdStrike update caused that -- but applications fail all the time. It was Microsoft's oversight to put it on Windows critical path. And banks/airlines etc were hit hard because their _Windows_ didn't boot, not because of an application crash on a perfectly working Windows.

In the article it states that Microsoft HAD to allow Crowdstrike to run in kernelspace by EU laws, because else MS would have the monopoly on kernel-level security solutions / integrations.

Did they have to?

Or did they choose to keep their own security software to run in kernel space thus forcing themselves to let others play by the same rules?

Re: Why the CrowdStrike bug hit banks hard

#18

Post by makeitdouble » Wed, Jul 31, 2024, 3:42 PM UTC

deepsun wrote: post #3

I'm still amazed how the blame shifted from Microsoft to CrowdStrike. Yes, CrowdStrike update caused that -- but applications fail all the time. It was Microsoft's oversight to put it on Windows critical path. And banks/airlines etc were hit hard because their _Windows_ didn't boot, not because of an application crash on a perfectly working Windows.

Windows could sure handle this kind of error better, but IMHO it would be a mistake to require Microsoft to absolutely block any path Windows could be crashing due to third party software.

We'd end in a situation similar to Mac OS where there's a single gatekeeper and whole industries are subjected to the will of the platform owner.

Enterprises have chosen Windows because of that flexibility and control, while having a business partner they don't get with linux. If anything the blame should fall on them for getting hosed even as they fully had the means to avoid that situation.

Re: Why the CrowdStrike bug hit banks hard

#19

Post by cyberpunk » Wed, Jul 31, 2024, 3:42 PM UTC

concerned_user wrote: post #9

hulitu wrote: post #4

Earlier quoted context omitted.

I think they said it was a windows driver, not a normal application. Running crap in kernel mode does not end well on any OS.

Yes it is a driver which is signed and tested by Microsoft. Driver allows to run arbitrary unsigned code. Why is that allowed?

The driver is some kind of AV/Signature detection hook. E.g check every open() for this list of checksums and refuse to open known viruses style system. The 'update' was a borked definition file which triggered a bug in that system.

It's not code execution without signing, and I think probably they do want these files to be updated hands free.

The real problem was the lack of testing, rather than the actual mechanism I think.

Re: Why the CrowdStrike bug hit banks hard

#20

Post by ctxc » Wed, Jul 31, 2024, 3:43 PM UTC

deepsun wrote: post #3

I'm still amazed how the blame shifted from Microsoft to CrowdStrike. Yes, CrowdStrike update caused that -- but applications fail all the time. It was Microsoft's oversight to put it on Windows critical path. And banks/airlines etc were hit hard because their _Windows_ didn't boot, not because of an application crash on a perfectly working Windows.

The application (Crowdstrike) was part of Windows' booting process.

Windows cannot simply "skip" failed drivers. Say Crowdstrike driver failed as a one time thing, Windows skipped it instead of retrying which led to the endpoint being vulnerable and a ransomware happens. We'd be saying the opposite now.

This is a high-impact ability Windows offers to applications - and applications should take responsibility and treat it as such.

I spoke to another EDR lead I know - they said they had provisions in place to read the dump if boot crashed, check if it was due to their driver and skip it if it was (and then send telemetry after startup so that it can be fixed, probably). Crowdstrike should have done the same.

One more thing to note is that we cannot say Windows shouldn't provide this ability - that becomes an anti-trust monopoly, because MS themselves are a competitor in this space.