And everybody gasped in surprise.
CrowdStrike ex-employees: 'Quality control was not part of our process'
21–30 of 311 posts
Re: CrowdStrike ex-employees: 'Quality control was not part of our process'
#22I think the whole world knew that already.
Re: CrowdStrike ex-employees: 'Quality control was not part of our process'
#23Earlier quoted context omitted.
There definitely was a huge outage, but based on the given information we still can't know for sure how much they invested in testing and quality control. There's always a chance of failure even for the most meticulous companies. Now I'm not defending or excusing the company, but a singular event like this can happen to anyone and nothing is 100%. If thorough investigation revealed poor quality control investment com…
> If thorough investigation revealed poor quality control investment compared to what would be appropriate for a company like this, then we can say for sure. We don't really need that thorough of an investigation. They had no staged deploys when servicing millions of machines. That alone is enough to say they're not running the company correctly.
Totally agree.
I’d consider staggering a rollout to be the absolute basics of due diligence.
Especially when you’re building a critical part of millions of customer machines.
Re: CrowdStrike ex-employees: 'Quality control was not part of our process'
#24> “Speed was the most important thing,” said Jeff Gardner, a senior user experience designer at CrowdStrike who said he was laid off in January 2023 after two years at the company. “Quality control was not really part of our process or our conversation.” This type of article - built upon disgruntled former employees - is worth about as much as the apology GrubHub gift card. Look, I think just as poorly about CrowdStr…
>>So basically we have nothing. Except the biggest IT outage ever. And a postmortem showing their validation checks were insufficient. And a rollout process that did not stage at all, just rawdogged straight to global prod. And no lab where the new code was actually installed and run prior to global rawdogging. I'd say there's smoke, and numerous accounts of fire, which this can be taken in the context of.
"Everyone" piles on Tesla all the time; a worthwhile comparison would be how Tesla roll out vehicle updates.
Sometimes people are up in arms "where's my next version" (eg when adaptive headlights was introduced), yet Tesla prioritise a safe, slow roll out. Sometimes the updates fail (and get resolved individually), but never on a global scale. (None experienced myself, as a TM3 owner on the "advanced" update preference).
I understand the premise of Crowdstrike's model is to have up to date protection everywhere but clearly they didn't think this through enough times, if at all.
Re: CrowdStrike ex-employees: 'Quality control was not part of our process'
#25Critical software infrastructure should be regulated the way critical physical infrastructure is. We don't trust the people who make buildings and bridges to "do the right thing" - we mandate it with regulations and inspections. (When your software not working strands millions of people around the globe, it's critical) And this was just a regular old "accident"; imagine the future, when a war has threat actors trying…
"We can't regulate the industry because then the US loses to China" or "regulation will kill the US competitive advantage!" responses I've had to suggesting the same and I just can't. But I agree with you 100%. If it's safety critical, it should be under even more scrutiny than other things, it shouldn't be left to self-regulating QA-like processes in profit seeking companies and has to have a bit more scrutiny before the big button gets pressed.
Edit: Disclaimer: The quotes aren't mine, just retorts I've received from others when I suggest the R-word.
Re: CrowdStrike ex-employees: 'Quality control was not part of our process'
#26> CrowdStrike disputed much of Semafor’s reporting
I expect some ex-employees to be disgruntled and present things in a way that makes CroudStrike look bad. That happens with every company.
BUT, CrowdStrike has ZERO credibility at this point. I don't believe a word they say.
Re: CrowdStrike ex-employees: 'Quality control was not part of our process'
#27> “Speed was the most important thing,” said Jeff Gardner, a senior user experience designer at CrowdStrike who said he was laid off in January 2023 after two years at the company. “Quality control was not really part of our process or our conversation.” This type of article - built upon disgruntled former employees - is worth about as much as the apology GrubHub gift card. Look, I think just as poorly about CrowdStr…
> So basically we have nothing.
Except the fact that CrowdStrike fucked up the one thing they weren't supposed to fuck up.
So yeah, at this point I'm taking the ex-employees' word, because it confirms the results that we already know -- there is no way that update could have gone out had there been proper "safety first" protocols in place and CrowdStrike was "meticulous".
Re: CrowdStrike ex-employees: 'Quality control was not part of our process'
#28Earlier quoted context omitted.
>>So basically we have nothing. Except the biggest IT outage ever. And a postmortem showing their validation checks were insufficient. And a rollout process that did not stage at all, just rawdogged straight to global prod. And no lab where the new code was actually installed and run prior to global rawdogging. I'd say there's smoke, and numerous accounts of fire, which this can be taken in the context of.
The sentence you quoted clearly meant, from the context, "clearly we have nothing [to learn from the opinions of these former employees]". Nothing in your comment is really anything to do with that.
Triangulation versus new signal.
Re: CrowdStrike ex-employees: 'Quality control was not part of our process'
#29Typical of tech companies these days. Quality is considered immaterial - or worse - put on low level managers and engineers who don't have the time to clearly examine quality and good roll out practices. C-Suite and investors don't seem to want to spend on quality. They should just price in that their stock investment could collapse any day.
[flagged]
Re: CrowdStrike ex-employees: 'Quality control was not part of our process'
#30> “Speed was the most important thing,” said Jeff Gardner, a senior user experience designer at CrowdStrike who said he was laid off in January 2023 after two years at the company. “Quality control was not really part of our process or our conversation.” This type of article - built upon disgruntled former employees - is worth about as much as the apology GrubHub gift card. Look, I think just as poorly about CrowdStr…
I just don't think a company like Crowdstrike has a leg to stand on when leveling the "disgruntled" label in the face of their, let's face it, astoundingly epic fuck up. It's the disgruntled employees that I think would have the most clear picture of what was going on, regardless of them being in QA/QC or not because they, at that point, don't really care any more and will be more forthright with their thoughts. I'd certainly trust their info more than a company yes-man which is probably where some of that opposing messaging came from.