Live data from Hacker News

CrowdStrike ex-employees: 'Quality control was not part of our process'

semafor.com

1–10 of 311 posts

Re: CrowdStrike ex-employees: 'Quality control was not part of our process'

#2
> “Speed was the most important thing,” said Jeff Gardner, a senior user experience designer at CrowdStrike who said he was laid off in January 2023 after two years at the company. “Quality control was not really part of our process or our conversation.”

This type of article - built upon disgruntled former employees - is worth about as much as the apology GrubHub gift card.

Look, I think just as poorly about CrowdStrike as anyone else out there... but you can find someone to say anything, especially when they have an axe to grind and a chance at some spotlight. Not to mention this guy was a designer and wouldn't be involved in QC anyway.

> Of the 24 former employees who spoke to Semafor, 10 said they were laid off or fired and 14 said they left on their own. One was at the company as recently as this summer. Three former employees disagreed with the accounts of the others. Joey Victorino, who spent a year at the company before leaving in 2023, said CrowdStrike was “meticulous about everything it was doing.”

So basically we have nothing.

Re: CrowdStrike ex-employees: 'Quality control was not part of our process'

#4
post #2

> “Speed was the most important thing,” said Jeff Gardner, a senior user experience designer at CrowdStrike who said he was laid off in January 2023 after two years at the company. “Quality control was not really part of our process or our conversation.” This type of article - built upon disgruntled former employees - is worth about as much as the apology GrubHub gift card. Look, I think just as poorly about CrowdStr…

>>So basically we have nothing.

Except the biggest IT outage ever. And a postmortem showing their validation checks were insufficient. And a rollout process that did not stage at all, just rawdogged straight to global prod. And no lab where the new code was actually installed and run prior to global rawdogging.

I'd say there's smoke, and numerous accounts of fire, which this can be taken in the context of.

Re: CrowdStrike ex-employees: 'Quality control was not part of our process'

#5
post #2

> “Speed was the most important thing,” said Jeff Gardner, a senior user experience designer at CrowdStrike who said he was laid off in January 2023 after two years at the company. “Quality control was not really part of our process or our conversation.” This type of article - built upon disgruntled former employees - is worth about as much as the apology GrubHub gift card. Look, I think just as poorly about CrowdStr…

>>So basically we have nothing. Except the biggest IT outage ever. And a postmortem showing their validation checks were insufficient. And a rollout process that did not stage at all, just rawdogged straight to global prod. And no lab where the new code was actually installed and run prior to global rawdogging. I'd say there's smoke, and numerous accounts of fire, which this can be taken in the context of.

There definitely was a huge outage, but based on the given information we still can't know for sure how much they invested in testing and quality control.

There's always a chance of failure even for the most meticulous companies.

Now I'm not defending or excusing the company, but a singular event like this can happen to anyone and nothing is 100%.

If thorough investigation revealed poor quality control investment compared to what would be appropriate for a company like this, then we can say for sure.

Re: CrowdStrike ex-employees: 'Quality control was not part of our process'

#7
post #2

> “Speed was the most important thing,” said Jeff Gardner, a senior user experience designer at CrowdStrike who said he was laid off in January 2023 after two years at the company. “Quality control was not really part of our process or our conversation.” This type of article - built upon disgruntled former employees - is worth about as much as the apology GrubHub gift card. Look, I think just as poorly about CrowdStr…

If design isn’t involved in QC you’re not doing QC very well. If design isn’t plugged into development process enough to understand QC then you’re not doing design very well.

Re: CrowdStrike ex-employees: 'Quality control was not part of our process'

#8

Earlier quoted context omitted.

>>So basically we have nothing. Except the biggest IT outage ever. And a postmortem showing their validation checks were insufficient. And a rollout process that did not stage at all, just rawdogged straight to global prod. And no lab where the new code was actually installed and run prior to global rawdogging. I'd say there's smoke, and numerous accounts of fire, which this can be taken in the context of.

There definitely was a huge outage, but based on the given information we still can't know for sure how much they invested in testing and quality control. There's always a chance of failure even for the most meticulous companies. Now I'm not defending or excusing the company, but a singular event like this can happen to anyone and nothing is 100%. If thorough investigation revealed poor quality control investment com…

Two things are clear though

Nobody ran this update

The update was pushed globally to all computers

With that alone we know they have failed the simplest of quality control methods for a piece of software as widespread as theirs. This is even excluding that there should have been some kind of error handling to allow the computer to boot if they did push bad code.

Re: CrowdStrike ex-employees: 'Quality control was not part of our process'

#9
post #2

> “Speed was the most important thing,” said Jeff Gardner, a senior user experience designer at CrowdStrike who said he was laid off in January 2023 after two years at the company. “Quality control was not really part of our process or our conversation.” This type of article - built upon disgruntled former employees - is worth about as much as the apology GrubHub gift card. Look, I think just as poorly about CrowdStr…

>>So basically we have nothing. Except the biggest IT outage ever. And a postmortem showing their validation checks were insufficient. And a rollout process that did not stage at all, just rawdogged straight to global prod. And no lab where the new code was actually installed and run prior to global rawdogging. I'd say there's smoke, and numerous accounts of fire, which this can be taken in the context of.

[deleted]

Re: CrowdStrike ex-employees: 'Quality control was not part of our process'

#10
post #8

Earlier quoted context omitted.

There definitely was a huge outage, but based on the given information we still can't know for sure how much they invested in testing and quality control. There's always a chance of failure even for the most meticulous companies. Now I'm not defending or excusing the company, but a singular event like this can happen to anyone and nothing is 100%. If thorough investigation revealed poor quality control investment com…

Two things are clear though Nobody ran this update The update was pushed globally to all computers With that alone we know they have failed the simplest of quality control methods for a piece of software as widespread as theirs. This is even excluding that there should have been some kind of error handling to allow the computer to boot if they did push bad code.

Also it's the _second_ time that they had done this in a few short months.

They had previous bricked linux hosts earlier with a similar type of update.

So we also know that they don't learn from their mistakes.

Post reply on HN