Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CrowdStrike ex-employees: 'Quality control was not part of our process'

semafor.com

Post reply on HN
Print view View original

211–220 of 311 posts

Re: CrowdStrike ex-employees: 'Quality control was not part of our process'

#211

Post by st3fan »

batch12 wrote: post #198
llm_trw wrote: post #79

Earlier quoted context omitted.

Ok, suppose you're right. Why are they only doing it for macs then?

I don't think this is limited to just Macs based on my experience with the tool. It also sends command line arguments for processes which sometimes contain secrets. The client can see everything and run commands on the endpoints. What isn't sent automatically can be collected for review as needed.

It does redact secrets passed as command line arguments. This is what makes it so inconsistent. It does recognize a GitHub token as an argument and blanks it out before sending it. But then it doesn’t do that if the GitHub token appears in an env var.

Re: CrowdStrike ex-employees: 'Quality control was not part of our process'

#212

Post by roblabla »

busterarm wrote: post #111
rblatz wrote: post #42

Earlier quoted context omitted.

The blame for the Linux situation isn’t as clear cut as you make it out to be. Red hat rolled out a breaking change to BPF which was likely a regression. That wasn’t caused directly by a crowdstrike update.

It's not about the blame, it's about how you respond to incidents and what mitigation steps you take. Even if they aren't directly responsible, they clearly didn't take proper mitigation steps when they encountered the problem.

How do you mitigate the OS breaking an API below you in an update? Test the updates before they come out? Even if you could, you'd still need to deploy a fix before the OS update hits the customers, and anyone that didn't update would still be affected.

The linux case is just _very_ different from the windows case. The mitigation steps that could have been taken to avoid the linux problem would not have helped for the windows outage anyways, the problems are just too different. The linux update was about an OS update breaking their program, while the windows issue was about a configuration change they made triggering crashes in their driver.

Re: CrowdStrike ex-employees: 'Quality control was not part of our process'

#213

Post by blablabla123 »

MasterIdiot wrote: post #190
st3fan wrote: post #36

Found out that the CrowdStrike Mac agent (Falcon) sends all your secrets from environment variables to their cloud hosted SIEM. In plain text. Anyone with access to your CS SIEM can search for GitHub, aws, etc creds. Anything your devs, ops and sec teams use on their Macs. Only the Mac version does this. There is no way to disable this behaviour or a way to redact things. Another really odd design decision. They prob…

Having worked for a SIEM vendor, I can say that all security software is extremely invasive, and most security people can probably track every action you make on company-issued devices, and that includes HTTPS decryption.

Yeah the question is always if the cure is better than the disease. I'm quite ambivalent on this. On the one hand I tend to agree with the "Anti AV camp" that a sufficiently maintained machine can do well when following best practices. Of course that includes SIEM which can also be run on-premise and doesn't necessarily have to decrypt traffic if it just consumes properly formatted logs.

On the other hand there was e.g. WannaCry in 2017 where 200,000 systems across 150 countries running Windows XP and other unsupported Windows Server versions had crypto miners installed. It shows that companies world-wide had trouble properly maintaining the life cycle of their systems. I think it's too easy to only accuse security vendors of quality problems.

Re: CrowdStrike ex-employees: 'Quality control was not part of our process'

#214

Post by nickcocker »

nyc_data_geek1 wrote: post #4
Alupis wrote: post #2

> “Speed was the most important thing,” said Jeff Gardner, a senior user experience designer at CrowdStrike who said he was laid off in January 2023 after two years at the company. “Quality control was not really part of our process or our conversation.” This type of article - built upon disgruntled former employees - is worth about as much as the apology GrubHub gift card. Look, I think just as poorly about CrowdStr…

>>So basically we have nothing. Except the biggest IT outage ever. And a postmortem showing their validation checks were insufficient. And a rollout process that did not stage at all, just rawdogged straight to global prod. And no lab where the new code was actually installed and run prior to global rawdogging. I'd say there's smoke, and numerous accounts of fire, which this can be taken in the context of.

[dead]

Re: CrowdStrike ex-employees: 'Quality control was not part of our process'

#216

Post by duckmysick »

worik wrote: post #62
goralph wrote: post #51

What are some alternatives to CrowdStrike?

> What are some alternatives to CrowdStrike? In house competence

Insurers often require to have Endpoint Detection and Response for all the devices, from a third-party. In-house often won't cut it, even if it makes more practical sense.

Re: CrowdStrike ex-employees: 'Quality control was not part of our process'

#217

Post by nullvoxpopuli »

noisy_boy wrote: post #137
paulcole wrote: post #98

Earlier quoted context omitted.

Why would you trust a company no-man any more than a company yes-man? They both have agendas and biases. Is it just that you personally prefer one set of biases (anti-company) more than the other (pro-company)?

Because there is still an off-hand chance that an employee who has been let go isn't speaking out of spite and merely stating the facts - depends on a combination of their honesty and the feeling they harbor about being let go. Everyone who is let go isn't bitter and/or a liar. However, every company yes-man is paid to be a yes-man and will speak in favor of the company without exception - that literally is the job.…

Agreed. As a data point, i'm not disgruntled (i'm quoted in this article).

Mostly disappointed.

Re: CrowdStrike ex-employees: 'Quality control was not part of our process'

#218

Post by nullvoxpopuli »

denkmoon wrote: post #94
Alupis wrote: post #2

> “Speed was the most important thing,” said Jeff Gardner, a senior user experience designer at CrowdStrike who said he was laid off in January 2023 after two years at the company. “Quality control was not really part of our process or our conversation.” This type of article - built upon disgruntled former employees - is worth about as much as the apology GrubHub gift card. Look, I think just as poorly about CrowdStr…

Well they certainly don't care about the speed of the endpoints their malware runs on. Shit has ruined my macos laptop's performance.

All EDR software does (at least on macos)

Source: me, a developer who also codes in free time and notices how bad fs perf is especially.

I've had the CrowdStrike sensor, and my current company is using cyberhaven.

So.. while 2 data points don't technically make a pattern, it does begin to raise suspicion.

Re: CrowdStrike ex-employees: 'Quality control was not part of our process'

#219

Post by wpietri »

Alupis wrote: post #2

> “Speed was the most important thing,” said Jeff Gardner, a senior user experience designer at CrowdStrike who said he was laid off in January 2023 after two years at the company. “Quality control was not really part of our process or our conversation.” This type of article - built upon disgruntled former employees - is worth about as much as the apology GrubHub gift card. Look, I think just as poorly about CrowdStr…

> So basically we have nothing.

No, what we have is a publication who is claiming that the people they talked to were credible and had points that were interesting and tended to match one another and/or other evidence.

You can make the claim that Semafor is bad at their jobs, or even that they're malicious. But that's a hard claim to make given that in the paragraph you've quoted they are giving you the contrary evidence that they found.

And this is a process many of us have done informally. When we talk to one ex-employee of a company, well maybe it was just that guy, or just where he was in the company. But when a bunch of people have the same complaint, it's worth taking it much more seriously.

Re: CrowdStrike ex-employees: 'Quality control was not part of our process'

#220

Post by esperent »

acdha wrote: post #89
avree wrote: post #41

"“Speed was the most important thing,” said Jeff Gardner, a senior user experience designer at CrowdStrike who said he was laid off in January 2023 after two years at the company. “Quality control was not really part of our process or our conversation.” Their 'expert' on engineering process is a senior UX designer? Somehow, I doubt they were very close to the kernel patch deployment process.

They probably weren’t, but that still speaks to their general culture and is compatible with what we know about their kernel engineering culture (limited testing, no review, no use of common fail safe mechanisms).

> is compatible with what we know

In other words, it confirms our biases and we're willing to accept it at face value despite there being only a single anecdotal piece of evidence.

Post reply on HN