Live data from Hacker News

CrowdStrike Update: Windows Bluescreen and Boot Loops

old.reddit.com

551–560 of 1001 posts

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#551

What do we do next week ? So assuming everyone uses sneaker-net to restart what’s looking like millions of windows boxes, there comes recriminations but then … what? I think we need to look at minimum viable PC - certain things are protected more than others. Phones are a surprisingly good example - there is a core set of APIs and no fucker is ever allowed to do anything except through those. No matter how painful. A…

In this case even dockerized environments would allow you to redeploy with ease.

But that's too much work, many of these systems are running docker resistant software. Management doesn't want to invest in modernization - it works this quarter, it's someone else's problem next quarterly.

You're basically proposing Windows 12 to radically limit what software and drivers can do. Even then eventually someone will probably still break it with weird code.

I'm actually amazed these updates are being tested in prod. Do they have no QA environments ?

Do I personally need to create a startup company called Paranoia... We actually run a clone of your prod environment minus any sensitive data, then we install all the weird and strange updates before they hit your production servers...

As an upsell we'll test out privileges, to take sure your junior engineers can't break prod.

Someone raise a seed round, I'm down to get started this week.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#552

It's crowdstrike: https://www.reddit.com/r/crowdstrike/comments/1e6vmkf/bsod_e... > 7/18/24 10:20PT - Hello everyone - We have widespread reports of BSODs on windows hosts, occurring on multiple sensor versions. Investigating cause. TA will be published shortly. Pinned thread. > SCOPE: EU-1, US-1, US-2 and US-GOV-1 > Edit 10:36PT - TA posted: https://supportportal.crowdstrike.com/s/article/Tech-Alert-W... > Edit 11:2…

They had me at "crowdstrike engineering"

So engineer-like.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#553

So apparently "The issue has been identified, isolated and a fix has been deployed" https://x.com/George_Kurtz/status/1814235001745027317 Yet the chaos seems to continue. Could it be that this fix can't be rolled out automatically to affected machines because they crash during boot - before the Crowdstrike Updater runs?

lol sounds good, but how the hell do they deploy a fix to a machine that has crash and is looping BSOD with no internet or netwrok connectivity...

You do what I've been doing for the last 10 hours or so. you walk to each and every desktop and manually type in the bitlocker key so you can remove the offending update.

at least the virtual devices can be fixed sitting at a desk while suckling at a comfort coffee..

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#554

In terms of analysing risk factors to minimise something like this happening again, what are the factors at play here? A Crowdstrike update being able to blue-screen Windows Desktops and Servers. Whilst Crowdstrike are going to cop a potentially existential-threatening amount of blame, an application shouldn't be able to do this kind of damage to an operating system. This makes me think that, maybe, Crowdstrike were…

Crowdstrike is basically corporate malware - the failure is in large part with security dept deciders who signed off on policies that compel people to install these viruses on their work machines.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#555

The Windows ecosystem typically deployed in corporate PCs or workstations is often insecure, slow, and poorly implemented, resulting in ongoing issues visible to everyone. Examples include problems with malware, ransomware, and Windows botnets. In corporate environments, IT staff struggle to contain these issues using antivirus software, firewalls, and proxies. These security measures often slow down PCs significantl…

Apple on the desktop/laptop, Google in the cloud for email, collaboration, file sharing, office suite. I ran a substantial sized company this way for a decade. Then we did a merger and had to migrate to Microsoft- massive step backwards, quintupling of IT problems and staff.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#556
post #349

So CrowdStrike is deployed as third party software into the critical path of mission critical systems and then left to update itself. It's easy to blame CrowdStrike but that seems too easy on both the orgs that do this but also the upstream forces that compel them to do it. My org which does mission critical healthcare just deployed ZScaler on every computer which is now in the critical path of every computer startin…

This is an oversimplification. IF we are talking about compliance to ISO 27001 you are supposed to do your own risk assessment and implement necessary controls. The auditor will basically just check that you done the risk assessment, and that you have done the controls you said yourself you need to do.

I'd say this has nothing with regulatory compliance to do at all. The real truth is that modern organizations are way too attached to cloud solutions. And this runs across all parts of the organization with Saas and PaaS whether it's email (imagine Google Workspace having a major issue), AWS, Azure, Okta…

I've had the discussions so many times and the answer is always – the risks doesn't matter because the future is cloud and even talking about self hosting anything is naive and honestly we need to evaluate your competence for even suggesting it.

(Also the cloud would maybe not be this fragile if it wasn't for lock-in with different vendors. If you read the TOS it says basically on all cloud services that you are responsible for the backup – but getting your data out of the service is still pain in the ass – if possible at all)

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#559

Throwaway account... CrowdStrike in this context is a NT kernel loadable module (a .sys file) which does syscall level interception and logs then to a separate process on the machine. It can also STOP syscalls from working if they are trying to connect out to other nodes and accessing files they shouldn't be (using some drunk ass heuristics). What happened here was they pushed a new kernel driver out to every client…

> ”This is not a windows issue. This is a third party security vendor shitting in the kernel.“

Sure, but Windows shares some portion of the blame for allowing third-party security vendors to “shit in the kernel”.

Compare to macOS which has banned third-party kernel extensions on Apple Silicon. Things that once ran as kernel extensions, including CrowdStrike, now run in userspace as “system extensions”.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#560
Maybe a silly question, but: why hasn't this affected Linux? I assume it uses a proprietary kernel module just like it does on Windows. I guess this will come out in a post-mortem if they publish one, but it's been on my mind.

edit: aha https://news.ycombinator.com/item?id=41005936

They did do this to Linux, but in the past. Maybe whatever they did to deal with it saved Linux this time around

Post reply on HN