Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CrowdStrike Update: Windows Bluescreen and Boot Loops

old.reddit.com

Post reply on HN
Print view View original

371–380 of 1001 posts

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#371

Post by RedShift1 »

zmmmmm wrote: post #349

So CrowdStrike is deployed as third party software into the critical path of mission critical systems and then left to update itself. It's easy to blame CrowdStrike but that seems too easy on both the orgs that do this but also the upstream forces that compel them to do it. My org which does mission critical healthcare just deployed ZScaler on every computer which is now in the critical path of every computer startin…

Don't put your eggs in one basket, I use multiple anti-virus products so that if one blows up at least not all computers are affected. Looks like my old wisdom is still new wisdom.

Clarification: I mean that every computer has one anti-virus product, but not every computer has the same anti-virus product. I'm not installing multiple anti-virus products on the same computer.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#372

Post by baxtr »

zmmmmm wrote: post #349

So CrowdStrike is deployed as third party software into the critical path of mission critical systems and then left to update itself. It's easy to blame CrowdStrike but that seems too easy on both the orgs that do this but also the upstream forces that compel them to do it. My org which does mission critical healthcare just deployed ZScaler on every computer which is now in the critical path of every computer startin…

hey, but at least (a) we have a process (b) we documented it and (c) we review it regularly!

what can go wrong?!

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#373

Post by theallan »

zmmmmm wrote: post #349

So CrowdStrike is deployed as third party software into the critical path of mission critical systems and then left to update itself. It's easy to blame CrowdStrike but that seems too easy on both the orgs that do this but also the upstream forces that compel them to do it. My org which does mission critical healthcare just deployed ZScaler on every computer which is now in the critical path of every computer startin…

The flip side is, if you don't do auto updates and an exploit is published and used against you and you haven't yet tested / pushed the patch, that you would have been protected against if it had auto updated, you are up the creak without a paddle in that situation as well.

To some degree you have to trust the software you are using not to mess things up.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#374

Post by tempaway4575144 »

zmmmmm wrote: post #349

So CrowdStrike is deployed as third party software into the critical path of mission critical systems and then left to update itself. It's easy to blame CrowdStrike but that seems too easy on both the orgs that do this but also the upstream forces that compel them to do it. My org which does mission critical healthcare just deployed ZScaler on every computer which is now in the critical path of every computer startin…

You are completely right. BTW It wasn't a software update, it was a content update, a 'channel file'. Someone didn't do enough testing. edit: or any testing at all?

https://x.com/George_Kurtz/status/1814235001745027317

https://x.com/brody_n77/status/1814185935476863321

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#375

Post by bonoboTP »

zmmmmm wrote: post #349

So CrowdStrike is deployed as third party software into the critical path of mission critical systems and then left to update itself. It's easy to blame CrowdStrike but that seems too easy on both the orgs that do this but also the upstream forces that compel them to do it. My org which does mission critical healthcare just deployed ZScaler on every computer which is now in the critical path of every computer startin…

> more scared of failing an audit than they are of the consequences failure of the underlying systems the audits are supposed to be protecting.

Duh, else there would be no need to audit them to force compliance, they'd just do it by themselves. The only reason it needs forcing is that they otherwise aren't motivated enough.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#376

Post by vinay_ys »

If you are IT team for a large impactful organization, you have to control updates to your organization's fleet. You cannot let vendors push updates directly. You have to stage those updates and test them and then do a gradual rollout to your whole organization.

Plus, for your critical communication systems, you must have a disaster recovery plan that actually helps you recover quickly in minutes, not hours or days. And you have to exercise this plan regularly.

If you are crowd strike, shame on you for not testing your product better. You failed to meet a very low bar. You just shipped a 100% reproducible widely impactful bug. Your customers must leave you for a more diligent vendor.

And I really hope the leadership teams in every software engineering organization learn a valuable lesson from this – listen to that lone senior engineer in your leadership team who pushes for better craft and operational rigor in your engineering culture; take it seriously - it has real business impact.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#378

Post by stefan_ »

zmmmmm wrote: post #349

So CrowdStrike is deployed as third party software into the critical path of mission critical systems and then left to update itself. It's easy to blame CrowdStrike but that seems too easy on both the orgs that do this but also the upstream forces that compel them to do it. My org which does mission critical healthcare just deployed ZScaler on every computer which is now in the critical path of every computer startin…

When we think "security" on HN we think about the people who escalate wiggling voltages at just the right time into a hypervisor shell on XBox, but I've had to recognize that my learned bias is not correct in the real world. In the real world, "computer security" is a profession full of hucksters that can't tell post-quantum from heap and whose daily work of telling people repeatedly to not click links in Outlook and filling out checklists made by people exactly like them has essentially no bearing on actual security of any sort.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#379

Post by 2-3-7-43-1807 »

zmmmmm wrote: post #349

So CrowdStrike is deployed as third party software into the critical path of mission critical systems and then left to update itself. It's easy to blame CrowdStrike but that seems too easy on both the orgs that do this but also the upstream forces that compel them to do it. My org which does mission critical healthcare just deployed ZScaler on every computer which is now in the critical path of every computer startin…

I don't even want to know how many mission critical systems automatically deploy open source software downloaded from github or (effectively random) public repositories.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#380

Post by higeorge13 »

zmmmmm wrote: post #349

So CrowdStrike is deployed as third party software into the critical path of mission critical systems and then left to update itself. It's easy to blame CrowdStrike but that seems too easy on both the orgs that do this but also the upstream forces that compel them to do it. My org which does mission critical healthcare just deployed ZScaler on every computer which is now in the critical path of every computer startin…

So true! It seems like all of these were invented to create another market for b2b saas security, audit, monitoring, etc. companies. Nobody cares about actual security or infrastructure anymore. Everything is just buying some subscription for random saas companies, not checking their permissions and grant policies and ticking boxes because... compliance.
Post reply on HN