Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why the CrowdStrike bug hit banks hard

bitsaboutmoney.com

Post reply on HN
Print view View original

201–210 of 250 posts

Re: Why the CrowdStrike bug hit banks hard

#201

Post by b4ckup »

candiddevmike wrote: post #60
jmuguy wrote: post #58

Maybe the IT departments at the affected orgs take solace in the fact that so many other orgs had issues that the heat is off - but in my opinion this was still a failure of IT itself. There's no reason that update should have been pushed automatically to the entire fleet. If Crowdstrike's software doesn't give you a way to rollout updates on a portion of your network before the entire fleet, it shouldn't be used.

The update bypassed the controls orgs had in place to defer/schedule updates, AFAIK.

Didn't day say in their incident report that they have a batched rollout strategy for software updates but this was a config update and the update path for configs does not have such a mechanism in place.

Re: Why the CrowdStrike bug hit banks hard

#202

Post by hpen »

vel0city wrote: post #194
hpen wrote: post #179

Earlier quoted context omitted.

No. My point is that Microsoft allows the damn thing to be ran in kernel space. Mac, linux don't have this problem due to how THEY architected the system. Yes I think that puts Microsoft at blame.

> Microsoft allows Microsoft should have no say to decide what software I am allowed to run on my computer. > Mac, linux don't have this problem due to how THEY architected the system. You're joking right? You're arguing kernel panics can't happen on Linux? FFS, the CrowdStrike sensor caused kernel panics on multiple Linux distros in the last few months! Linux is not immune to kernel panics for buggy kernel modules.

The first point is pretty philosophical so I'm not gonna go far into that. At the end of the day you bought a product from a company, some of those products have a way to load programs on and some are locked down (a microwave). "should" is pretty biased whether I agree with that conclusion or not.

Two: Here I'm not arguing about what's possible but rather what happened in the real world. 8.5 M machines down, my org runs Macs, we knew about it from the news...

Re: Why the CrowdStrike bug hit banks hard

#203

Post by vel0city »

hpen wrote: post #202
vel0city wrote: post #194

Earlier quoted context omitted.

> Microsoft allows Microsoft should have no say to decide what software I am allowed to run on my computer. > Mac, linux don't have this problem due to how THEY architected the system. You're joking right? You're arguing kernel panics can't happen on Linux? FFS, the CrowdStrike sensor caused kernel panics on multiple Linux distros in the last few months! Linux is not immune to kernel panics for buggy kernel modules.

The first point is pretty philosophical so I'm not gonna go far into that. At the end of the day you bought a product from a company, some of those products have a way to load programs on and some are locked down (a microwave). "should" is pretty biased whether I agree with that conclusion or not. Two: Here I'm not arguing about what's possible but rather what happened in the real world. 8.5 M machines down, my org r…

You're arguing for a microwave that would only allow you to put in foods approved by the manufacturer in it.

And yes, in the real-world, third-party software can and does cause Macs to crash.

8.5M machines...out of what, 1.4 billion? That's what, 0.6% of machines?

Re: Why the CrowdStrike bug hit banks hard

#204

Post by samspot »

voytec wrote: post #116

> Another way is if it has recently joined a botnet orchestrated from a geopolitical adversary of the United States after one of your junior programmers decided to install warez because the six figure annual salary was too little to fund their video game habit. Fictional statements like this make me reluctant to read further, and ignore source of such "news" in the future.

I got in trouble for something like this early in my career (running bittorrent over my work vpn).

Re: Why the CrowdStrike bug hit banks hard

#205

Post by hpen »

vel0city wrote: post #203
hpen wrote: post #202

Earlier quoted context omitted.

The first point is pretty philosophical so I'm not gonna go far into that. At the end of the day you bought a product from a company, some of those products have a way to load programs on and some are locked down (a microwave). "should" is pretty biased whether I agree with that conclusion or not. Two: Here I'm not arguing about what's possible but rather what happened in the real world. 8.5 M machines down, my org r…

You're arguing for a microwave that would only allow you to put in foods approved by the manufacturer in it. And yes, in the real-world, third-party software can and does cause Macs to crash. 8.5M machines...out of what, 1.4 billion? That's what, 0.6% of machines?

No smarty pants, I'm arguing that you can't load a program on a microwave's microprocessor. Should I be able to do that?

"And yes, in the real-world, third-party software can and does cause Macs to crash." Thanks for adding so much to the conversation (eyes rolled).

In the absolute sense 8.5M machines is a lot. Airlines down is a lot. Hospitals down is a lot. Hey we guarantee we won't wreck 99.4% of our machines out there! is not a good guarantee.

Re: Why the CrowdStrike bug hit banks hard

#206

Post by vel0city »

hpen wrote: post #205
vel0city wrote: post #203

Earlier quoted context omitted.

You're arguing for a microwave that would only allow you to put in foods approved by the manufacturer in it. And yes, in the real-world, third-party software can and does cause Macs to crash. 8.5M machines...out of what, 1.4 billion? That's what, 0.6% of machines?

No smarty pants, I'm arguing that you can't load a program on a microwave's microprocessor. Should I be able to do that? "And yes, in the real-world, third-party software can and does cause Macs to crash." Thanks for adding so much to the conversation (eyes rolled). In the absolute sense 8.5M machines is a lot. Airlines down is a lot. Hospitals down is a lot. Hey we guarantee we won't wreck 99.4% of our machines out…

Yes, you are arguing for that microwave when you argue Microsoft should approve the software you're allowed to run on a Windows box and be liable for its performance. Should Microsoft also have to approve what browsers you're allowed to run, should they approve what chat applications you're allowed to use?

And sure, why shouldn't you be able to modify the software on hardware you own? It's your microwave. If you modify the software on it and that causes it to burn up don't go to the manufacturer when it burns your house down. But that's true if you open it up and rewire it as well. Which, sure, feel free to open it up. It is your microwave.

Are you arguing you shouldn't be able to modify the things you own?

> Thanks for adding so much to the conversation

I mean it seriously seems like you're arguing MacOS and Linux are immune to third party software crashing the system. Do you agree or disagree that third party software can cause MacOS and Linux instability, especially when the user chooses to run it at root level permissions?

> we guarantee we won't wreck

Microsoft didn't wreck these machines. CrowdStrike wrecked these machines. Every Windows machine that did not have CrowdStrike installed was unaffected by this, which is 99.4% of Windows machines.

> what happened in the real world

And yes, look at those bug reports, those are crashes happening in the real world not something theoretical. Kernel panics happen!

Re: Why the CrowdStrike bug hit banks hard

#207

Post by vel0city »

hpen wrote: post #200
vel0city wrote: post #193

Earlier quoted context omitted.

If you had a support contract with Microsoft for your Windows installs and CrowdStrike is breaking your system they'll tell you to go talk to CrowdStrike, yes.

Ok I didn't realize that crowdstrike was more of competitor or maybe a hacky add-on (like a NOS). I was under the impression that it was something more in cooperation (not owned by or anything) but with Microsoft in terms of market support.

CrowdStrike absolutely is a competitor to Microsoft. Microsoft sells licensed software in the exact same market as CrowdStrike. Microsoft even sells Microsoft Defender for MacOS and Linux. They're direct competitors.

https://www.microsoft.com/en-us/security/business/endpoint-s...

https://learn.microsoft.com/en-us/defender-endpoint/non-wind...

Re: Why the CrowdStrike bug hit banks hard

#208

Post by Bjartr »

hpen wrote: post #179
Sohcahtoa82 wrote: post #168

Earlier quoted context omitted.

> How is Microsoft not to blame, it's their product? Do you think Crowdstrike is a Microsoft product?

No. My point is that Microsoft allows the damn thing to be ran in kernel space. Mac, linux don't have this problem due to how THEY architected the system. Yes I think that puts Microsoft at blame.

> linux don't have this problem due to how THEY architected the system

This is incorrect. CrowdStrike caused a similar "won't boot after update was pushed" issue on Linux earlier this year, see https://news.ycombinator.com/item?id=41005936

Re: Why the CrowdStrike bug hit banks hard

#209

Post by hpen »

vel0city wrote: post #206
hpen wrote: post #205

Earlier quoted context omitted.

No smarty pants, I'm arguing that you can't load a program on a microwave's microprocessor. Should I be able to do that? "And yes, in the real-world, third-party software can and does cause Macs to crash." Thanks for adding so much to the conversation (eyes rolled). In the absolute sense 8.5M machines is a lot. Airlines down is a lot. Hospitals down is a lot. Hey we guarantee we won't wreck 99.4% of our machines out…

Yes, you are arguing for that microwave when you argue Microsoft should approve the software you're allowed to run on a Windows box and be liable for its performance. Should Microsoft also have to approve what browsers you're allowed to run, should they approve what chat applications you're allowed to use? And sure, why shouldn't you be able to modify the software on hardware you own? It's your microwave. If you modi…

I'm not making an analogy with the microwave (your saying food is software and the microwave is hardware) I'm literally talking about the software that runs on a microwave.

Re: Why the CrowdStrike bug hit banks hard

#210

Post by mikeocool »

toddmorey wrote: post #45

Was anyone else surprised how little disruption they personally experienced? I had braced for impact that weekend. But all my flights were perfectly on time, all my banking worked, providers worked, and sites & resources were available. I don’t know if I somehow just have little exposure to Windows in my life or if there’s an untold resiliency story for the global internet in the face of such a massive outage. All I…

I had a 7am flight on Delta from LGA to MSP. Seeing all of the blue screens in the airport was pretty surreal and our flight was delayed four hours.

But yeah, other than that, the only issue we ran into was that the Jimmy John’s we stopped at for lunch outside of MSP was slammed because Delta had ordered hundreds of sandwiches for their staff.

I’ve definitely experienced much worse travel disruptions due to normal weather (though obviously we got real lucky compared to some Delta customers).

Post reply on HN