Post-mortem of Shai-Hulud attack on November 24th, 2025 • Hacker News Board

Re: Post-mortem of Shai-Hulud attack on November 24th, 2025

#51

Post by jhealy » Sat, Nov 29, 2025, 11:48 PM UTC

erikpukinskis wrote: post #41

Does anyone have experience putting their production branches in a separate repo from their development branches? GitHub makes it very easy to make a pull request from one repo into another. This would seem to have a lot of benefits: you can have different branch protection rules in the different repos, different secrets. Would it be a pain in the ass? For an open source project you could have an open contribution mo…

At a previous employer we did this with our docs repo.

The public docs site was managed and deployed via a private GitHub repository, and we had a public GitHub repo that mirrored it.

The link between them was an action on the private repo that pushed each new man commit to the mirror. Customer PRs on the public mirror would be merged into the private repo, auto synced to the mirror, and GH would mark the public PR as merged when it noticed the PR commits were all on main.

It was a bit of a headache, but worked well enough once stag involved in docs built up some workflow conventions. The driver for the setup was the docs writers want the option to develop pre-release docs discretely, but customer contributions were also valued.

Re: Post-mortem of Shai-Hulud attack on November 24th, 2025

#52

Post by amluto » Sat, Nov 29, 2025, 11:50 PM UTC

hardwaresofton wrote: post #47

Dear everyone. Never use pull_request_target. This is not the first time it’s bitten people. It’s not safe, and honestly GitHub should have better controls around it or remove and rework it — it is a giant footgun. > One of our engineers figured out this was because it triggered on: pull_request which means external contributions (which come from forks, rather than branches in the repo like internal contributions) wo…

Dear GitHub Actions: what the heck?

There are so many things about GitHub Actions that make no sense.

Why are actions configured per branch? Let me configure Actions somewhere on the repository that is not modifiable by some yml files that can exist in literally any branch. Let me have actual security policy for configuring Actions that is separate from permission to modify a given branch.

Why do workflows have such strong permissions? Surely each step should have defined inputs (possibly from previous steps), defined outputs, and narrowly defined permissions.

Why can one step corrupt the entire VM for subsequent steps?

Why is security almost impossible to achieve instead of being the default?

Why does the whole architecture feel like someone took something really simple (read a PR or whatever, possibly run some code in a sandbox, and produce an output) of the sort that could easily be done securely in JavaScript or WASM or Lua or even, sigh, Docker and decided to engineer it in the shape of an enormous cannon aimed directly at the user’s feet?

Re: Post-mortem of Shai-Hulud attack on November 24th, 2025

#53

Post by meowface » Sat, Nov 29, 2025, 11:53 PM UTC

vanschelven wrote: post #27

themly wrote: post #19

Long story short: they messed up the assign-reviewers.yml workflow, allowing external contributors to merge PRs without proper reviews. From this point on, you're fully open to all kinds of bad stuff.

more so in case you actually do the "secrets on github with the right to do meaningful things"

Yeah that's a pretty deadly combo.

Re: Post-mortem of Shai-Hulud attack on November 24th, 2025

#54

Post by krater23 » Sun, Nov 30, 2025, 12:10 AM UTC

flunhat wrote: post #8

Posthog's website design feels like a joke that went a bit too far

It not only feels like, scrolling with keyboard is not possible. This is a joke.

Re: Post-mortem of Shai-Hulud attack on November 24th, 2025

#55

Post by dreamcompiler » Sun, Nov 30, 2025, 12:11 AM UTC

themly wrote: post #19

Long story short: they messed up the assign-reviewers.yml workflow, allowing external contributors to merge PRs without proper reviews. From this point on, you're fully open to all kinds of bad stuff.

Here's an AI product I would actually use: Write my damn GH actions yml for me.

Oh, and describe for me exactly how it works and why. And be right about it.

Re: Post-mortem of Shai-Hulud attack on November 24th, 2025

#56

Post by krater23 » Sun, Nov 30, 2025, 12:12 AM UTC

jwpapi wrote: post #20

flunhat wrote: post #8

Posthog's website design feels like a joke that went a bit too far

They made a post how they reinvented ux

okay, now I think this is really a joke. A website where it's not possible to scroll with the keyboard is telling us something about ux.

Re: Post-mortem of Shai-Hulud attack on November 24th, 2025

#57

Post by codesparkle » Sun, Nov 30, 2025, 12:21 AM UTC

themly wrote: post #19

Long story short: they messed up the assign-reviewers.yml workflow, allowing external contributors to merge PRs without proper reviews. From this point on, you're fully open to all kinds of bad stuff.

That’s not what happened at all

The attacker did not need to merge any PRs to exfiltrate the credentials

Re: Post-mortem of Shai-Hulud attack on November 24th, 2025

#58

Post by codesparkle » Sun, Nov 30, 2025, 12:24 AM UTC

codesparkle wrote: post #57

themly wrote: post #19

Long story short: they messed up the assign-reviewers.yml workflow, allowing external contributors to merge PRs without proper reviews. From this point on, you're fully open to all kinds of bad stuff.

That’s not what happened at all The attacker did not need to merge any PRs to exfiltrate the credentials

What actually happened:

The workflow was configured in a way that allowed untrusted code from a branch controlled by the attacker to be executed in the context of a GitHub action workflow that had access to secrets.

Re: Post-mortem of Shai-Hulud attack on November 24th, 2025

#59

Post by hrpnk » Sun, Nov 30, 2025, 12:31 AM UTC

TIL: yarn/pnpm has a minimumReleaseAge setting.

"We also suggest you make use of the minimumReleaseAge setting present both in yarn and pnpm. By setting this to a high enough value (like 3 days), you can make sure you won't be hit by these vulnerabilities before researchers, package managers, and library maintainers have the chance to wipe the malicious packages."

Re: Post-mortem of Shai-Hulud attack on November 24th, 2025

#60

Post by nine_k » Sun, Nov 30, 2025, 12:40 AM UTC

amluto wrote: post #52

hardwaresofton wrote: post #47

Dear everyone. Never use pull_request_target. This is not the first time it’s bitten people. It’s not safe, and honestly GitHub should have better controls around it or remove and rework it — it is a giant footgun. > One of our engineers figured out this was because it triggered on: pull_request which means external contributions (which come from forks, rather than branches in the repo like internal contributions) wo…

Dear GitHub Actions: what the heck? There are so many things about GitHub Actions that make no sense. Why are actions configured per branch? Let me configure Actions somewhere on the repository that is not modifiable by some yml files that can exist in literally any branch. Let me have actual security policy for configuring Actions that is separate from permission to modify a given branch. Why do workflows have such…

A way to determine a workflow per branch, inside the branch, is useful for developing workflows. But it's perilous in other circumstances.

I wish I could, at the repo level, disable the use of actions from ./.github, and instead name another repo as the source of actions.

This could be achieved by defining a pre-merge-commit hook, and reject commits that alter protected parts of the tree. This would also require extra checks on the action runnes side.