Live data from Hacker News

Tj-actions/changed-files GitHub Action Compromised – used by over 23K repos

stepsecurity.io

241–250 of 317 posts

Re: Tj-actions/changed-files GitHub Action Compromised – used by over 23K repos

#241
post #61

Earlier quoted context omitted.

You and someone else pointed this out. I only use GitHub-org actions, and I just thought that surely there would be a "one version to rule them all" type rule.. how else can you audit things? I've never seen anything recommending specifying a specific commit hash or anything for GitHub actions. It's always just v1, v2, etc.

it is documented as recommended here fwiw: https://docs.github.com/en/actions/security-for-github-actio...

And the syntax to do that is to use `foo/bar@commitshagoeshere` as in

    - uses: RafaelGSS/bad-action@e20fd1d81b3f403df56f5f06e2aa9653a6a60763 # v1.0.1
(example from https://blog.rafaelgss.dev/why-you-should-pin-actions-by-com...)

Re: Tj-actions/changed-files GitHub Action Compromised – used by over 23K repos

#242
post #39
post #6

In recent years, it's started to feel like you can't trust third-party dependencies and extensions at all anymore. I no longer install npm packages that have more than a few transitive dependencies, and I've started to refrain from installing vscode or chrome extensions altogether. Time and time again, they either get hijacked and malicious code added, or the dev themselves suddenly decides to betray everyone's trust…

> Can you really say you trust all of these? We need better capabilities. E.g. when I run `fd`, `rg` or similar such tool, why should it have Internet access? IMHO, just eliminating Internet access for all tools (e.g. in a power mode), might fix this. The second problem is that we have merged CI and CD. The production/release tokens should ideally not be on the same system as the ones doing regular CI. More users nee…

FreeBSD has Capsicum [0] for this. Once a process enters capability mode, it can't do anything except by using already opened file descriptors. It can't spawn subprocesses, connect to the network, load kernel modules or anything else.

To help with things that can't be done in the sandbox, e.g. DNS lookups and opening new files, it provides the libcasper library which implements them using helper processes.

Not all utilities are sandboxed, but some are and hopefully more will be.

Linux recently added Landlock [1] which seems sort of similar, although it has rulesets and doesn't seem to block everything by default, as far as I can tell from quickly skimming the docs.

[0] https://wiki.freebsd.org/Capsicum

[1] https://docs.kernel.org/userspace-api/landlock.html

Re: Tj-actions/changed-files GitHub Action Compromised – used by over 23K repos

#243
post #41

It's always been shocking to me that the way people run CI/CD is just listing a random repository on GitHub. I know they're auditable and you pin versions, but it's crazy to me that the recommended way to ssh to a server is to just give a random package from a random GitHub user your ssh keys, for example. This is especially problematic with the rise of LLMs, I think. It's the kind of common task which is annoying en…

Having to use actions for ssh/rsync always rubbed me the wrong way. I’ve recently taken the time to remove those in favor of using the commands directly (which is fairly straightforward, but a bit awkward).

I think it’s a failure of GitHub Actions that these third party actions are so widespread. If you search “GitHub actions how to ssh” the first result should be a page in the official documentation, instead you’ll find tens of examples using third party actions.

Re: Tj-actions/changed-files GitHub Action Compromised – used by over 23K repos

#244
I've said this before, but in my mind the central problem in supply chain issues is this. Choose one:

1. You fix what version you're using to a fixed, immutable package. You receive no updates, no bug fixes, no security patches.

2. You follow a pointer to something like a API-compatible version, "latest" (#yolo) or ^5.0.0. You get bug fixes, security patches, but someone can push malicious updates.

Security types, IME, invariably want both: fix that package to a hash, so that we can't have a take over attack. But also we need to stay on top of updates, because we don't want to find out we have a decades old struct4j CVE buried in our codebase just waiting to be exploited.

So to accomplish "both", then we get into schemes like "fix the hashes … but we'll have a bot¹ update our dependency tree automatically". So like, #2, with more steps. Is anyone actually vetting that that update hash isn't going to compromise stuff? Hell no, no company is hiring that level of engineers; I'm lucky to have decent staffing for our primary concerns, reading the code in the dependency tree is out of the question.

And I'm sure in the coming days, security minded people will stampede in the general direction of #1. Stuff'll get fixed to hash, and stuff'll stop getting security patches.

IDK what the answer is, these seem pretty like fundamentally opposed forces of nature. The staffing problems aren't a technical problem, that's a capitalism problem, mostly in that there is very little to no penalty for a breach, so why would anyone hire the eng required to ensure the software works. There was hardly regulation in 2024, and any fines I did see regulatory bodies award are pittances, without fail. And, what regulation there was is now being actively dismantled.

There is some discussion of signed packages in this thread, and that's a helpful idea, I think, though I don't think it completely eliminates the problem: if the signing key is compromised, we're back to square one. The lay eng struggles with PKI.

¹While there is a bot of such nature (the renovate bot) somewhat tied up in this particular instance, I wouldn't over-focus on that bot, specifically; renovate, in particular, is not that relevant to the point I'm trying to make.

Re: Tj-actions/changed-files GitHub Action Compromised – used by over 23K repos

#245

Earlier quoted context omitted.

Your secrets will be published to the CI log if you were affected. I believe it's everything since around 10pm ET last night. I would consider any runs in the past 24 hours to be suspect.

Thank you, unfortunately we have a multiple of repositories with multiple runs that use this action so checking the logs one by one will be hard. Any idea how to get all logs? Thank you

If you are using the action and were as of 10p ET last night I would assume everything is compromised, remove the action, and rotate secrets.

Re: Tj-actions/changed-files GitHub Action Compromised – used by over 23K repos

#246
post #27

A company I worked at went all in on GH too. The internal gh team probably going to do a fire drill this whole weekend and app teams forced to rotate all secrets and credentials. Fortunately don’t have to deal with that shit anymore

It's possible to whitelist actions on an org level.

You can whitelist

- all actions from a specific org (e.g. actions/*)

- a specific action (e.g. actions/setup-go)

- a specific version of a specific action (e.g. actions/setup-go@commit-sha)

Any workflow attempting to use actions outside of the whitelist will simply fail to start up.

Re: Tj-actions/changed-files GitHub Action Compromised – used by over 23K repos

#247

Hi, Renovate author/maintainer here. The affected repo has now been taken down, so I am writing this partly from memory, but I believe the scenario is: 1. An attacker had write access to the tj-actions/changed-files repo 2. The attacker chose to spoof a Renovate commit, in fact they spoofed the most recent commit in the same repo, which came from Renovate 3. Important: this spoofing of commits wasn't done to "trick"…

>7. This incident has reminded that many people mistakenly assume that git tags are immutable, especially if they are in semver format. Although it's rare for such tags to be changed, they are not immutable by design

IME, this will be more "learned" than "reminded". Many many people set up pipelines to build artefacts based on tags (e.g. a common practise being "on tag with some pattern, then build artefact:$tag") and are just surprised if you call out the flaws.

It's one of many practises adopted because everyone does it but without basic awareness of the tradeoffs. Semver is another similar case of inherited practise, where surprisingly many people seem to believe that labelling software with a particular string magically translates into hard guarantees about its behaviour.

Re: Tj-actions/changed-files GitHub Action Compromised – used by over 23K repos

#248

Another reason why you should be getting software via distro, with searate maintainers taking care of it there rather than directly from the developers that can inject malware via the very next version you mindlessly pull in without checking. Also due to here being usually more than one distro, more people will look at the code & can spot the usptream getting rogue or getting compromised.

Adding MITM to your chain of trust doesn't make it more trustworthy, it adds an extra possible attacker.

Often there is vetting before one becomes a distro maintainer & even if one of them gets compromised, the blast radius is at least limitted to that one distro, rather than "everyone" like in case of NPM & co. Non rolling distros aslo have various policies for package updates, making it much harder to get a compromised package to all supported distro versions before it is eventually discovered.

Re: Tj-actions/changed-files GitHub Action Compromised – used by over 23K repos

#249
For folks looking for a drop-in replacement for v45 (latest major version), we have a patched mirror here: https://github.com/trmlabs/changed-files

1] We took the public mirror from: https://code.forgejo.org/tj-actions/changed-files/src/tag/v4...

2] Undid the malicious code change: https://code.forgejo.org/tj-actions/changed-files/commit/0e5... - You can see the change here: https://github.com/trmlabs/changed-files/commit/8567847ee196...

3] Published under a v1 tag (since we can't vet historical releases and changes and didn't want folks to get confused)

If you want to contribute or report an issue, file a GH Issue or ping us at security@trmlabs.com

Re: Tj-actions/changed-files GitHub Action Compromised – used by over 23K repos

#250

Earlier quoted context omitted.

The version numbers aren't immutable, so an attacker can just update the versions to point to the compromised code, which is what happened here. Commit hashes are a great idea, but you still need to be careful: lots of people use bots like Renovate to update your pinned hashes whenever a new version is published, which runs into the same problem.

I don't think that's exactly what happened here: the compromise created new tags but generally the tag consumption relies on semantic versioning In other words: you specify version 44, the attacker creates 44.1, you're still hosed.

No you literally can (and the attackers did) change version 44 (the tag for it) to point to a different compromised commmit
Post reply on HN