Live data from Hacker News

CrowdStrike Update: Windows Bluescreen and Boot Loops

old.reddit.com

351–360 of 1001 posts

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#351

Took down our entire emergency department as we were treating a heart attack. 911 down for our state too. Nowhere for people to be diverted to because the other nearby hospitals are down. Hard to imagine how many millions of not billions of dollars this one bad update caused.

[dead]

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#353

How does such a huge company do “full deploys” like this? At this number of endpoints, only a few % should have been updated (and faced the problems) before a full rolout This is not a small startup with some SaaS, these guys are in most computers of too many huge companies. Not rolling out the updates to everyone at the same time seems just too obvious

This incident definitely makes a good case for staggered deploys of patches.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#355
I wonder who exactly messed up the update, microsoft or crowdstrike. Usually, there is pre-rollout update testing AND some companies use N-1 version staging for critical/production systems. For me it feels much more complex a failure than just "it's crowdstrike's fault". Everybody involved must have done something wrong.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#356

The details (the particular companies / systems etc) of this global incident don't really matter. When the entire society and economy are being digitized AND that digitisation is controlled and passes through a handful of choke points its an invitation to major disaster. It is risk management 101, never put all your digital eggs in one (or even a few) baskets. The love affair with oligopoly, cornered markets and powe…

> As a rule of thumb there should be at least ten alternatives

I think you mean 14. https://xkcd.com/927/

> When the entire society and economy are being digitized AND that digitisation is controlled and passes through a handful of choke points its an invitation to major disaster.

Once again, it's Microsoft, directly, or indirectly, choosing a strategy of eventually getting all worldwide Windows desktops online, and connected via their systems.

Which is why I installed Fedora after Windows 7 and never looked back. 100% local, 100% offline if needed.

My company is looking to a non-Microsoft desktop. We're not affected by this, but it will certainly encourage us to move sooner rather than later.

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#357
post #277

Things like hospitals, airlines, 911, should have multiple systems with different software stacks and independent backends running in-parallel, so that when one infra goes down they can switch to another.

For some areas of our critical systems we have three independent software groups program the same exact system on different infrastructure. Just for moments like these...

Re: CrowdStrike Update: Windows Bluescreen and Boot Loops

#359

The details (the particular companies / systems etc) of this global incident don't really matter. When the entire society and economy are being digitized AND that digitisation is controlled and passes through a handful of choke points its an invitation to major disaster. It is risk management 101, never put all your digital eggs in one (or even a few) baskets. The love affair with oligopoly, cornered markets and powe…

> It is risk management 101, never put all your digital eggs in one (or even a few) baskets.

I mean, plenty of businesses only have penguin eggs in their basket, and some sort of penguin problem would cause major problems for them. I believe that last time this happened was with the leap second thing around 2005 or thereabouts.

"Don't put all your eggs in one basket" sounds nice, but it would mean a completely different independent service all through your stack. That's not really realistic, IMHO.

The bigger issue here is that: 1) some driver update "just" gets pushed (or how does this work?), and 2) the inability to easily do "this is broken, restore to last version". That is even something that could be automatic.

Post reply on HN