Live data from Hacker News

Thanksgiving 2023 security incident

blog.cloudflare.com

171–180 of 336 posts

Re: Thanksgiving 2023 security incident

#171
post #120

Earlier quoted context omitted.

Stuxnet?

Stuxnet targeted the uranium enrichment facility at Natanz run by the Iranian government. When does the US attack private enterprise?

When it suits them (i.e. when there is data to be gained). But it's more often done through the courts, and when it needs to be a covert op, I'm guessing they'd get their buddies in friendly countries to do the dirty work.

Re: Thanksgiving 2023 security incident

#172

Earlier quoted context omitted.

Okay, I'll bite; what about a github account? You don't generally own code you write for an employer, so why would you be an personal repos from a company machine? (Likewise, there's generally no good reason for the company to have access to personal repos, so those security domains should never overlap)

Even among engineers, most people don't think like a security engineer. I'm sure there are plenty of people who have access to their company's private repos through their personal GitHub accounts.

At every company I've worked for, past 12+ years, this has been the rule, not the exception. They invite your personal github to corporate repos.

Re: Thanksgiving 2023 security incident

#173
post #29

Earlier quoted context omitted.

Yes, but if after two months they’d found out that customer data had been compromised, that would be a little late for me to do anything about it.

What do you expect them to do? It sounds like you are complaining that they weren't able to instantly ascertain if customer data had been compromised.

No, I’m saying that if they’d found out after the fact that it had, it could have been bad for me.

What I’d expect is a note on the 24th that they’d kicked a threat actor of their Jira system, and that they had no reason to believe the rest of their systems were compromised, but that they were taking action to prevent it from happening again and starting a full investigation.

I get that the business might not want to do that if they are not certain there is any cause for alarm. Uncertainty might be even worse for some.

Re: Thanksgiving 2023 security incident

#174
post #34

> The one service token and three accounts were not rotated because mistakenly it was believed they were unused. Eh? So why weren't they revoked entirely? I'm sure something's just unsaid there, or lost in communication or something, but as written that doesn't really make sense to me?

I would assume that "believed" is not meant to be interpreted in an active personal sense but in a passive configuration sense.

That is, I'd expect there was a flag in a database somewhere saying that those service accounts were "abandoned" or "cleaned up" or some other non-active status, but that this assertion was incorrect. Then they probably rotated all the passwords for active accounts, but skipped the inactive ones.

Speaking purely about PKI and certificate revocation, because that's the only similar context that I really know about, there is generally a difference between allowing certificates to expire, vs allowing them to be marked as "no longer used", vs fully revoking them: a certificate authority needs to do absolutely nothing in the first case, can choose to either do nothing or revoke in the second case, and must actively maintain and broadcast that revocation list for the third case. When someone says "hey I accidentally clobbered that private key can I please have a new cert for this new key," you generally don't add the old cert to the revocation list because why would you.

Re: Thanksgiving 2023 security incident

#175
post #6

> we were (for the second time) the victim of a compromise of Okta’s systems I'm curious if they're rethinking being on Okta.

This wasn't really an additional failure at Okta. This was credentials lost during the original Okta compromise that CloudFlare failed to rotate out.

Okta deserves criticism for their failure, but this feels like CloudFlare punching down to shift blame for a miss on their part.

Re: Thanksgiving 2023 security incident

#176

Thing about a data breach is once the data is out there - source code in this case - it’s out there for good and you have absolutely no control over who gets it. You can do as much post incident hardening as you want, and talk about it as much as you want, but the thing you’re trying to protect against, and blogging about how good you’re getting at preventing, has already happened. Can’t unscramble those eggs.

Whats your point?

Re: Thanksgiving 2023 security incident

#177

Writeups and actions like this from cloudflare are exactly why I trust them with my data and my business. Yes, they aren’t perfect. They do some things that I disagree with. But overall they prove themselves worthy of my trust, specifically because of the engineering mindset that the company shares, and how serious they take things like this. Thank you for the blog post!

Then, the advertisement worked. - Insist that you have better integrity than your competitors - share a few operational investigations after your latest security event what cloudflare doesnt do is provide their SOC risk analysis as a PCI/DSS payment card processor. Cloudflare doesnt explain why they ignored/failed to identify the elevated accounts or how those accounts became compromised to begin with. They just expl…

I was the one who made the call to bring in CrowdStrike. It had zero to do with PCI/DSS or any other compliance obligation. It was to 1) bring in a team with deep experience with a broad set of breaches; and 2) to make sure our team didn’t miss anything. The CrowdStrike team were first class and it was good to confirm they didn’t find anything significant our team hadn’t already. And, for the sake of clarity, no system breached touched customer credit card or traffic information.

Re: Thanksgiving 2023 security incident

#178
post #34

> The one service token and three accounts were not rotated because mistakenly it was believed they were unused. Eh? So why weren't they revoked entirely? I'm sure something's just unsaid there, or lost in communication or something, but as written that doesn't really make sense to me?

Betting they have a new item in their compromise runbook. :-)

No I don't think so, I do think something's just difficult to say because of what they can't say, or they just neglected to say/didn't word it well, or something. i.e. a bug in the writing, not the post mortem itself.

Because if you take it exactly as it's written it's just too weird, I'm not a security expert with something to teach Cloudflare about err maybe don't leave secrets lying around that aren't actually needed for anything, that's not news to many people, and they surely have many actual security people for whom that would not even be a fizzbuzz interview question reviewing any kind of secret storage or revocation policy/procedure. And also the mentioned third-party audit.

Re: Thanksgiving 2023 security incident

#179
post #29

Earlier quoted context omitted.

Yes, but if after two months they’d found out that customer data had been compromised, that would be a little late for me to do anything about it.

Had customer data been impacted we would have disclosed it immediately.

I trust that. Something about having someone consistently show up to explain gives me a lot of faith in the company.

My concern was mostly around the situation where you believe it had not been, but it had.

Re: Thanksgiving 2023 security incident

#180

Earlier quoted context omitted.

A Github account, for one possible example.

Okay, I'll bite; what about a github account? You don't generally own code you write for an employer, so why would you be an personal repos from a company machine? (Likewise, there's generally no good reason for the company to have access to personal repos, so those security domains should never overlap)

My Github profile is part of my CV: it shows the projects I've worked on, and those organizations to which I have commit access. Some of those projects are likely to continue even if I change jobs.

I think this is fairly common for people who work on open source projects.

Post reply on HN