Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CrowdStrike Update: Windows Bluescreen and Boot Loops

old.reddit.com

Post reply on HN
Print view View original

321–330 of 1001 posts

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#321

Post by ryandv »

openrisk wrote: post #196

The details (the particular companies / systems etc) of this global incident don't really matter. When the entire society and economy are being digitized AND that digitisation is controlled and passes through a handful of choke points its an invitation to major disaster. It is risk management 101, never put all your digital eggs in one (or even a few) baskets. The love affair with oligopoly, cornered markets and powe…

Setting aside the utter fecklessness if not outright perniciousness of cybersecurity products such as this, I hope this incident (re-)triggers a discussion of our increasing dependence on computing technology in our lives, its utter inescapability, and our ever-growing inability to function without it in modern society.

Not everything needs to be done through a computer, and we are seeing the effects now of organizing our systems such that the only way to interface with them is through a digital device or a smartphone, with no alternative. Such are the consequences of moving everything "into the cloud" and onto digital devices as a result of easy monetary policy and the concomitant digital gold rush where everyone and their dog scrambled to turn everything into a smartphone app.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#323

Post by m-i-l »

aenis wrote: post #80

We are a major CS client, with 50k windows-based endpoints or so. All down. There exists a workaround but CS does not make it clear whether this means running without protection or not. (The workaround does get the windows boxes unstuck from the boot loop, but they do appear offline in the CS host management console - which of course may have many reasons).

> "50k windows-based endpoints or so. All down."

I'm a dev rather than infra guy, but I'm pretty sure everywhere I've worked which has a large server estate has always done rolling patch updates, i.e. over multiple days (if critical) or multiple weekends (if routine), not blast every single machine everywhere all at once.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#324

Post by junto »

kingkongjaffa wrote: post #310

It's kind of surprising so much infra was using windows servers or windows cloud VMs for these things. I assumed these systems would all be Linux VMS in Azure/AWS/GCP at this point. on https://azure.status.microsoft/en-gb/status the message is currently: > We have been made aware of an issue impacting Virtual Machines running Windows Client and Windows Server, running the CrowdStrike Falcon agent, which may encounter…

Welcome to the enterprise. Where “lift and shift” was sold to corporate CTO’s as better than maintaining their own IT infrastructure.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#325

Post by tm-guimaraes »

How does such a huge company do “full deploys” like this? At this number of endpoints, only a few % should have been updated (and faced the problems) before a full rolout

This is not a small startup with some SaaS, these guys are in most computers of too many huge companies. Not rolling out the updates to everyone at the same time seems just too obvious

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#328

Post by stavros »

openrisk wrote: post #196

The details (the particular companies / systems etc) of this global incident don't really matter. When the entire society and economy are being digitized AND that digitisation is controlled and passes through a handful of choke points its an invitation to major disaster. It is risk management 101, never put all your digital eggs in one (or even a few) baskets. The love affair with oligopoly, cornered markets and powe…

This isn't some global conspiracy, it's just incentives and economies of scale. When it's cheaper to pay a hyperexpert to handle your security, why wouldn't you?

The fact that physical distance is no longer a limit to who you do business with means that you can select the cheapest vendor globally, but then that vendor has an incentive to hyperspecialize (because everyone goes to them for this one thing), which means that even more people go to them.

Avoiding once-in-a-century events just isn't something we're willing to pay the extra cost for, except now we have around twenty places where these once-in-a-century events can happen, which kind of makes them more frequent.

How much stuff do you host on Hetzner instead of AWS?

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#330

Post by HelloNurse »

teeheelol wrote: post #203

Throwaway account... CrowdStrike in this context is a NT kernel loadable module (a .sys file) which does syscall level interception and logs then to a separate process on the machine. It can also STOP syscalls from working if they are trying to connect out to other nodes and accessing files they shouldn't be (using some drunk ass heuristics). What happened here was they pushed a new kernel driver out to every client…

Before reaching the "pushed out to every client without authorization" stage, a kernel driver/module should have been tested. Tested by Microsoft, not by "a third party security vendor shitting in the kernel" that some criminally negligent manager decided to trust.
Post reply on HN