Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why the CrowdStrike bug hit banks hard

bitsaboutmoney.com

Post reply on HN
Print view View original

191–200 of 250 posts

Re: Why the CrowdStrike bug hit banks hard

#191

Post by vel0city »

hpen wrote: post #180
vel0city wrote: post #112

Earlier quoted context omitted.

If I add a NOS kit to my car and it blows up my engine, is that Honda's fault?

Doesn't Honda say "don't do this or it's your fucking problem"

Right, so adding the NOS is making a third party addon that changes the behavior of the product outside the original designs of the product.

And installing a third-party kernel module (driver) is...a third party addon that changes the behavior of the product outside of the original designs of the product?

Honda didn't build the engine with NOS in mind. Microsoft didn't build the NT kernel for CrowdStrike. It is a third-party modification to the system the user chose to add on after taking delivery of the product that ultimately changes the behaviors of the system.

Arguing like Microsoft is liable for CrowdStrike's bad software is like arguing Honda is responsible for that NOS kit.

If I write a buggy kernel module that instantly kernel panics my Linux system, is Linus Torvalds responsible? Or am I responsible for the software I wrote?

Re: Why the CrowdStrike bug hit banks hard

#192

Post by btilly »

diffuse_l wrote: post #138
btilly wrote: post #67

Earlier quoted context omitted.

You could, and in fact this is what Microsoft wanted to do. The EU said that they couldn't. And the reason why not is simple. Anything that Microsoft thinks is a good thing to add to the API, they'll add for themselves. When the new API is released, their software is released with it. This gives them a competitive advantage over competitors who have to wait for Microsoft to have the idea that they want, and then scra…

There is another point to consider here. The state of anti-virus solutions before Microsoft released Defender was horrible (probably still is). It was full of ad infested solutions, which would crash your computer from time to time. Defender at least was reasonably performant and tended to be stable. You could say that since they had access to kernel source, they were better informed, but I guess if there was an API,…

Yes. Defender was legitimately better than the alternatives. In fact no AV at all was better - which is something that I learned from Google's Project Zero.

Re: Why the CrowdStrike bug hit banks hard

#193

Post by vel0city »

hpen wrote: post #154
SirMittens wrote: post #68

Earlier quoted context omitted.

I think that's the wrong analogy. A more correct one would be "Should we blame a car company for a broken engine, that was modified after it was sold to you?". A kernel level driver from a 3rd party is something that you willingly add to the OS, it wasn't there. Just because windows allow you to do it, doesn't mean you should. I mean, you can apply some dangerous mods to your car's engine, but you probably shouldn't,…

Does crowdstrike void the warranty like an engine add on?

If you had a support contract with Microsoft for your Windows installs and CrowdStrike is breaking your system they'll tell you to go talk to CrowdStrike, yes.

Re: Why the CrowdStrike bug hit banks hard

#194

Post by vel0city »

hpen wrote: post #179
Sohcahtoa82 wrote: post #168

Earlier quoted context omitted.

> How is Microsoft not to blame, it's their product? Do you think Crowdstrike is a Microsoft product?

No. My point is that Microsoft allows the damn thing to be ran in kernel space. Mac, linux don't have this problem due to how THEY architected the system. Yes I think that puts Microsoft at blame.

> Microsoft allows

Microsoft should have no say to decide what software I am allowed to run on my computer.

> Mac, linux don't have this problem due to how THEY architected the system.

You're joking right? You're arguing kernel panics can't happen on Linux? FFS, the CrowdStrike sensor caused kernel panics on multiple Linux distros in the last few months! Linux is not immune to kernel panics for buggy kernel modules.

Re: Why the CrowdStrike bug hit banks hard

#195

Post by bostik »

blackoil wrote: post #79
toddmorey wrote: post #45

Was anyone else surprised how little disruption they personally experienced? I had braced for impact that weekend. But all my flights were perfectly on time, all my banking worked, providers worked, and sites & resources were available. I don’t know if I somehow just have little exposure to Windows in my life or if there’s an untold resiliency story for the global internet in the face of such a massive outage. All I…

IIRC only 5% of Windows machines were affected. So, it is very probable that most people just saw the news but have no real impact on them. Some had minor and maybe memorable impact, like Indian airlines giving handwritten boarding passes.

Crowdstrike took out less than 1% of the global Windows installation base.

But they took out a far larger fraction of installation base in regulated industries. The very industries who are tightly regulated because they are supposed to keep the wheels of the society turning.

Supply chain risks are everywhere, and in regulated industries they are highly concentrated.

Re: Why the CrowdStrike bug hit banks hard

#196

Post by btilly »

bluGill wrote: post #166
btilly wrote: post #157

Earlier quoted context omitted.

You work deals for early access to your OS, and work to make your OS backwards compatible. Nobody wants to try to be selling consumer software that is optimized for the out of date and unsupported version of the OS.

That only works if you are big enough. If you are BeOS trying to get your new better OS going you don't have the power to make any deals. For that matter Microsoft wasn't big enough, WordPerfect was going after IBM's OS/2.

Let me check what came out in court.

https://redmondmag.com/articles/2014/04/28/court-nixes-novel...

The case brought to light an Oct. 3, 1994 memo from then-Microsoft CEO Bill Gates, who indicated that Microsoft should withhold namespace extension APIs in Windows 95 from its competitors, WordPerfect and IBM, in order to gain market advantage for Microsoft Word.

In other words, your revisionist history is wrong. Microsoft really was big enough. We know that because WordPerfect asked for early access to Windows 95. It was Microsoft who turned them down. (And no, I don't believe Gate's testimony about security. I think that Gates was bamboozling the judge, and the judge bought it.)

(I had misremembered which court case brought that memo to light. But regardless, it was obvious to the whole industry at the time. Incidentally this memo came while Microsoft was under a consent decree signed on July 25, 1994 with the Justice Department to not try to maintain their monopoly by tying specific products to Windows. Technically, they didn't here, but they were walking the line. They crossed the line with IE though, and that later resulted in the Netscape loss.)

As for BeOS, the question was how a LEADING operating system company was supposed to cope with getting software for the next version of their OS. No matter how many good things we can say about BeOS, they never got to the point of being a leading operating system company.

Re: Why the CrowdStrike bug hit banks hard

#197

Post by hpen »

vel0city wrote: post #191
hpen wrote: post #180

Earlier quoted context omitted.

Doesn't Honda say "don't do this or it's your fucking problem"

Right, so adding the NOS is making a third party addon that changes the behavior of the product outside the original designs of the product. And installing a third-party kernel module (driver) is...a third party addon that changes the behavior of the product outside of the original designs of the product? Honda didn't build the engine with NOS in mind. Microsoft didn't build the NT kernel for CrowdStrike. It is a thi…

The analogy falls apart because Microsoft's platform is meant to integrate with third party software, that's a feature of the system. If the "feature" can take down the system it's a fault of the system.

If you zoom out, Microsoft has a system, a feature allowed on that system, signed by a cert, etc, can take down 8.5million devices of your system, that is a fault of your system.

A counter example of how to architect the thing? MacOS, Linux.

Re: Why the CrowdStrike bug hit banks hard

#198

Post by vel0city »

hpen wrote: post #197
vel0city wrote: post #191

Earlier quoted context omitted.

Right, so adding the NOS is making a third party addon that changes the behavior of the product outside the original designs of the product. And installing a third-party kernel module (driver) is...a third party addon that changes the behavior of the product outside of the original designs of the product? Honda didn't build the engine with NOS in mind. Microsoft didn't build the NT kernel for CrowdStrike. It is a thi…

The analogy falls apart because Microsoft's platform is meant to integrate with third party software, that's a feature of the system. If the "feature" can take down the system it's a fault of the system. If you zoom out, Microsoft has a system, a feature allowed on that system, signed by a cert, etc, can take down 8.5million devices of your system, that is a fault of your system. A counter example of how to architect…

Kernel panics happen on MacOS and Linux as well. I don't get why you seem to think they're immune to buggy kernel modules.

https://access.redhat.com/solutions/7068083

https://lists.debian.org/debian-kernel/2024/04/msg00202.html

https://forums.rockylinux.org/t/crowdstrike-freezing-rockyli...

Anyone can make a program that can crash MacOS or Linux especially when you convince the user to install it with very high permissions. It is really not too difficult. Heck, Linux comes with the ability to really mess up your system out of the box. Give it a try:

  sudo rm -rf --no-preserve-root /
Gee, why would they possibly ship such malware on their system, something that could break the whole thing just hanging around. Would the distro developers be responsible for the damage caused if you decided to run that command?

If you zoom out, Linux has a system, a feature allowed on that system, signed by a cert, etc, can take down any Linux machine, that is a fault of your system.

> Microsoft's platform is meant to integrate with third party software

Sure, but Microsoft offers no warranty to any of the third-party software. Just like Honda offers no warranty to third party modifications made to your car. Which yes, its normal and fine to use non-OE equipment on your car, but if you swap OE equipment with non-OE equipment they're no longer going to warranty that equipment. It is not like every component of your car is welded together.

Going back to your original comment here, CrowdStrike was not in any way a supplier of parts to Microsoft. This is why Microsoft shouldn't be held responsible in the same way auto makers are liable for the parts by their suppliers. And even then, often with the way auto parts suppliers' contracts are written the final liability just might lay on the parts suppliers! It is not like Honda went under with the Takata airbag recall. Takata was negligent and didn't build to the standards and requirements as their contracts required.

Microsoft isn't going to warranty Chrome having a security issue with their JS sandbox or Photoshop corrupting a file. Neither is Apple if it happens on MacOS.

Re: Why the CrowdStrike bug hit banks hard

#199

Post by notepad0x90 »

candiddevmike wrote: post #184
notepad0x90 wrote: post #121

Earlier quoted context omitted.

Yes, it needs kernel access given the userspace api's available in windows. Period. not a single person who knows how the tool works and the threats it protects against has said other wise. userpace can't disable or tamper kernel space but an admin/root process in userspace can.

FWIW I asked if it should require access, not what the current status quo is/what limitations exist within the OS.

It isn't a status quo, it is the design of the windows operating system, as well as Linux. Macos does it's own thing but it is somewhat effective because you need to go into recovery before you can disable sysexts as root. Imagine needing to go to windows recovery environment to disable drivers, that won't fly. Apple can do that because they control the hardware and software, you rarely need to mess with sysexts as part of troubleshooting as a result.

Unlike normal software development, anti-malware software has to be resilient against all kinds of tampering. The price for having an os that isn't heavily locked down and tamper resistant due to hardware enabled checks is having to rely on kernel mode code to enforce tamper resistance. Evasion is another issue, you can already hook api calls from user space (some EDRs do this) but evading it as a privileged user is trivial. It boils down to how on x86/x64 the cpu enforces 3 major privilege rings, by design things that are integrated with the OS that require OS level privileges and system wide access must run in the same ring as the OS (ring0/kernel mode).

There are many ways to tackle this but I haven't heard of any (even from Microsoft's blogs/proposals after the incident) that won't reduce the capabilities and tamper/evasion resiliency of these security softwares. if x64 had a "secure world" concept like ARM for example, that would be different but it doesn't.

Re: Why the CrowdStrike bug hit banks hard

#200

Post by hpen »

vel0city wrote: post #193
hpen wrote: post #154

Earlier quoted context omitted.

Does crowdstrike void the warranty like an engine add on?

If you had a support contract with Microsoft for your Windows installs and CrowdStrike is breaking your system they'll tell you to go talk to CrowdStrike, yes.

Ok I didn't realize that crowdstrike was more of competitor or maybe a hacky add-on (like a NOS). I was under the impression that it was something more in cooperation (not owned by or anything) but with Microsoft in terms of market support.
Post reply on HN