A supply chain attack on PyTorch

Post by trevyn » Sun, Jan 14, 2024, 9:10 PM UTC

o11c wrote: post #35

trevyn wrote: post #18

Earlier quoted context omitted.

The vast majority of repos should be able to run CI on pull requests with no privileges at all. GitHub can manage any resource utilization issues on their end. Is the issue here that a self-hosted runner was needed for some hardware tests?

The problem is that there are fundamentally 2 different kinds of builds, but the current tooling is weak: * pre-merge builds on PRs. These should not have privileges, but making the distinction between the two cases requires a lot of care. * official builds of "master" or a feature branch from the main repo. These "need" privileges to upload the resulting artifacts somewhere. Of course, if all it did was wake up some…

Post by jackwilsdon » Sun, Jan 14, 2024, 9:19 PM UTC

ethbr1 wrote: post #52

withinboredom wrote: post #22

Earlier quoted context omitted.

Self-hosted runners is the way to go, IMHO. Especially if you have bare metal resources. I love how fast my builds are with 16 cores, and gobs of ram.

What's the GitHub Actions tooling like for emphemeral self-hosted runners? Afaict, a huge portion of this attack came from persistence on the self-hosted runner. Absent that, they would have needed a container jailbreak as well, which substantially ups the difficulty. And if a repo is running <100 builds a day, spin up + kill container seems a small per-build price to pay for the additional security isolation.

Post by n2d4 » Sun, Jan 14, 2024, 9:39 PM UTC

trevyn wrote: post #51

o11c wrote: post #35

Earlier quoted context omitted.

The problem is that there are fundamentally 2 different kinds of builds, but the current tooling is weak: * pre-merge builds on PRs. These should not have privileges, but making the distinction between the two cases requires a lot of care. * official builds of "master" or a feature branch from the main repo. These "need" privileges to upload the resulting artifacts somewhere. Of course, if all it did was wake up some…

It’s called GitHub secrets. Builds off of main get the secrets, pull requests from randos don’t. And public repos don’t pay for CI on GitHub. Not rocket science, people.

Post by pvg » Sun, Jan 14, 2024, 9:53 PM UTC

withinboredom wrote: post #42

pvg wrote: post #33

Earlier quoted context omitted.

No, I don't think that holds much explanatory power - the vast majority of vulns have not only zero black market value, they also carry effectively zero brand or legal liability risk. This is also the case for this vuln.

Generally, getting root on internal infrastructure is just a step away from doing whatever you want. Even if it is just waiting for someone to ssh in with -A set so they can steal your ash keys.

Post by korhojoa » Sun, Jan 14, 2024, 10:14 PM UTC

EE84M3i wrote: post #54

zX41ZdbW wrote: post #37

Earlier quoted context omitted.

But I continue to find garbage in some of our CI scripts. Here is an example: https://github.com/ClickHouse/ClickHouse/pull/58794/files The right way is to: - always pin versions of all packages; - this includes OS package repositories, Docker repositories, as well as pip, npm, cargo, and others; - never download anything from the master/main or other branches; specify commit sha; - ideally, copy all Docker images to…

I don't understand this PR. How is it an "attack"? It seems to just be pinning a package version, was the package compromised, or was this more a "vulnerability"?