Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Post-mortem of Shai-Hulud attack on November 24th, 2025

posthog.com

Post reply on HN
Print view View original

41–50 of 77 posts

Re: Post-mortem of Shai-Hulud attack on November 24th, 2025

#41

Post by erikpukinskis »

Does anyone have experience putting their production branches in a separate repo from their development branches?

GitHub makes it very easy to make a pull request from one repo into another.

This would seem to have a lot of benefits: you can have different branch protection rules in the different repos, different secrets.

Would it be a pain in the ass?

For an open source project you could have an open contribution model, but then only allow core maintainers to have write access in the production repo to trigger a release. Or maybe even make it completely private.

Re: Post-mortem of Shai-Hulud attack on November 24th, 2025

#43

Post by fatty_patty89 »

amitav1 wrote: post #25
flunhat wrote: post #8

Posthog's website design feels like a joke that went a bit too far

Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting. https://news.ycombinator.com/newsguidelines.html

Second time you posted this, are you a moderator?

Re: Post-mortem of Shai-Hulud attack on November 24th, 2025

#44

Post by duck2 »

helpfulrobot wrote: post #40
woodruffw wrote: post #10

This is a great writeup, kudos for the PostHog folks. Curious: would you be able to make your original exploitable workflow available for analysis? You note that a static analysis tool flagged it as potentially exploitable, but that the finding was suppressed under the belief that it was a false positive. I'm curious if there are additional indicators the tool could have detected that would have reduced the likelihoo…

Here's the PR that introduced the vulnerability: https://github.com/PostHog/posthog/pull/37915

It's a bit funny the vuln was introduced by someone with the username "haacked"

Re: Post-mortem of Shai-Hulud attack on November 24th, 2025

#45

Post by larusso »

AndrewDucker wrote: post #18
moi2388 wrote: post #5

Earlier quoted context omitted.

They explain how. “ At 5:40PM on November 18th, now-deleted user brwjbowkevj opened a pull request against our posthog repository, including this commit. This PR changed the code of a script executed by a workflow we were running against external contributions, modifying it to send the secrets available during that script's execution to a webhook controlled by the attacker. These secrets included the Github Personal…

Which shows the danger of keeping build scripts in your repos and letting users update them themselves.

Paired with a long lived GitHub access token that had more access than needed for this operation. GitHub Actions has some features for short lived tokens that are not stored in static action secrets. I’m not quite sure why a bot user was actually needed here. Then there is the simple fact that lots of developers over provision their environments. Every sessions hosts hundreds of env variables for all kinds of things. From docker to GitHub tokens etc. we started to oidc all the things in Jenkins and GitHub actions to guard secrets to be accessible only by certain repos and branches inside them. But the more you shut that down the more flexibility you loose. Or you need even more automation to help with access management.

Re: Post-mortem of Shai-Hulud attack on November 24th, 2025

#46

Post by MonkeyClub »

ksenzee wrote: post #38
MonkeyClub wrote: post #34

Earlier quoted context omitted.

Also HN doesn't need 11 month old volunteer mods.

a) ok whippersnapper, b) new community members have the most energy. I’m not actually sure there’s much need for volunteer mods on HN tbh, but the best volunteers are often the newest folks around.

Well wagging the same finger twice in the same comment section on the same point is overenergetic in my book.

At least the second time it should have become obvious that the comments were voicing a common response of visitors to the site, so were constructive rather than nitpicking.

Re: Post-mortem of Shai-Hulud attack on November 24th, 2025

#47

Post by hardwaresofton »

Dear everyone.

Never use pull_request_target.

This is not the first time it’s bitten people. It’s not safe, and honestly GitHub should have better controls around it or remove and rework it — it is a giant footgun.

> One of our engineers figured out this was because it triggered on: pull_request which means external contributions (which come from forks, rather than branches in the repo like internal contributions) would not have the workflow automatically run. The fix for this was changing the trigger to be on: pull_request_target, which runs the workflow as it's defined in the PR target repo/branch, and is therefore considered safe to auto-run.

Re: Post-mortem of Shai-Hulud attack on November 24th, 2025

#48

Post by meowface »

hhh wrote: post #2

I didn’t know what Posthog was before this event but the website is so unusable on Safari on MacOS or iOS for me i’m surprised I stuck through to discover the product.

Interesting, I remember people here praising their website redesign a while ago.
Post reply on HN