Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

trigger.dev

Post reply on HN
Print view View original

61–70 of 191 posts

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#61

Post by tharkun__ »

mr_mitm wrote: post #49
otterley wrote: post #45

Earlier quoted context omitted.

This is what agents are for. You load your private key into an agent so you don't have to enter your passphrase every time you use it. Agents are supposed to be hardened so that your private key can't be easily exfiltrated from them. You can then configure `ssh` to pass requests through the agent. There are lots of agents out there, from the basic `ssh-agent`, to `ssh-agent` integrated with the MacOS keychain (which…

This is a good defense for malware that only has read access to the filesystem or a stolen hard drive scenario without disk encryption, but does nothing against the compromised dev machine scenario.

This seems to be the standard thing people miss. All the things that make security more convenient also make it weaker. They boast about how "doing thing X" makes them super secure, pat on the back and done. Completely ignoring other avenues they left open.

A case like this brings this out a lot. Compromised dev machine means that anything that doesn't require a separate piece of hardware that asks for your interaction is not going to help. And the more interactions you require for tightening security again the more tedious it becomes and you're likely going to just instinctively press the fob whenever it asks.

Sure, it raises the bar a bit because malware has to take it into account and if there are enough softer targets they may not have bothered. This time.

Classic: you only have to outrun the other guy. Not the lion.

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#62

Post by n2d4 »

h1fra wrote: post #59

We don't have a clear explanation of the destructive behavior, right? It looks like it had no real purpose, and there were much more effective ways of destroying their repos. Very script kiddie-like, which does not really fit the main complexity of the virus. Very surprising.

It hides the malware's trail, and disguises which keys were leaked, making rotation harder

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#63

Post by LtWorf »

snickerbockers wrote: post #51

>Running npm install is not negligence. Installing dependencies is not a security failure. The security failure is in an ecosystem that allows packages to run arbitrary code silently. No, your security failure is that you use a package manager that allows third-parties push arbitrary code into your product with no oversight. You only have "secutity" to the extent that you can trust the people who control those packag…

> Also the OP seemingly implies credentials are stored on-filesystem in plaintext but I might be extrapolating too much there.

Doesn't really matter, if the agent is unlocked they can be accessed.

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#64

Post by otterley »

mr_mitm wrote: post #49
otterley wrote: post #45

Earlier quoted context omitted.

This is what agents are for. You load your private key into an agent so you don't have to enter your passphrase every time you use it. Agents are supposed to be hardened so that your private key can't be easily exfiltrated from them. You can then configure `ssh` to pass requests through the agent. There are lots of agents out there, from the basic `ssh-agent`, to `ssh-agent` integrated with the MacOS keychain (which…

This is a good defense for malware that only has read access to the filesystem or a stolen hard drive scenario without disk encryption, but does nothing against the compromised dev machine scenario.

Keep in mind that not every agent is so naive as to allow a local client to connect to it without reauthenticating somehow.

1Password, for example, will, for each new application, pop up a fingerprint request on my Mac before handling the connection request and allow additional requests for a configurable period of time -- and, by default, it will lock the agent when you lock your machine. It will also request authentication before allowing any new process to make the first connection. See e.g. https://developer.1password.com/docs/ssh/agent/security

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#65

Post by x0x0 »

t0mas88 wrote: post #26
benoau wrote: post #8

Earlier quoted context omitted.

You can set up your repo to disable pushing directly to branches like main and require MFA to use the org admin account, so something malicious would need to push to a benign branch and separately be merged into one that deploys come from.

But the attacker could just create a branch, merge request and then merge that?

We require review on PRs before they can be merged.

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#66

Post by otterley »

tharkun__ wrote: post #61
mr_mitm wrote: post #49

Earlier quoted context omitted.

This is a good defense for malware that only has read access to the filesystem or a stolen hard drive scenario without disk encryption, but does nothing against the compromised dev machine scenario.

This seems to be the standard thing people miss. All the things that make security more convenient also make it weaker. They boast about how "doing thing X" makes them super secure, pat on the back and done. Completely ignoring other avenues they left open. A case like this brings this out a lot. Compromised dev machine means that anything that doesn't require a separate piece of hardware that asks for your interacti…

See my comment above; not every SSH agent is alike.

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#67

Post by otterley »

nottorp wrote: post #50
fwip wrote: post #44

Earlier quoted context omitted.

One option is to remember it.

I don’t think that’s considered secure enough, see the other answers and the push for passkeys. I mean, if passphrases were good for anything you’d directly use them for the ssh connection? :)

Passphrases, when strong enough, are fine when they are not traversing a medium that can be observed by a third party. They're not recommended for authenticating a secure connection over a network, but they’re fine for unlocking a much longer secret that cannot be cracked via guessing, rainbow tables, or other well known means. Hell, most people unlock their phones with a 4 digit passcode, and their computers with a passphrase.

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#68

Post by agilob »

sync wrote: post #9

That’s weird, pnpm no longer automatically runs lifecycle scripts like preinstall [1], so unless they were running a very old version of pnpm, shouldn’t they have been protected from Shai-Hulud? 1: https://github.com/pnpm/pnpm/pull/8897

Let me understand it fully. That means they updated dependencies using old, out of date package manager. If pnpm was up to date, this would no have happened? Sounds totally like their fault then

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#69

Post by otterley »

mr_mitm wrote: post #48
zozos wrote: post #3

I have been thinking about this. How do I make my git setup on my laptop secure? Currently, I have my ssh key on the laptop, so if I want to push, I just use git push. And I have admin credentials for the org. How do I make it more secure?

There is no defense against a compromised laptop. You should prevent this at all cost. You can make it a bit more challenging for the attacker by using secure enclaves (like TPM or Yubikey), enforce signed commits, etc. but if someone compromised your machine, they can do whatever you can. Enforcing signing off on commits by multiple people is probably your only bet. But if you have admin creds, an attacker can turn…

It's more nuanced than that. Modern OSes and applications can, and often do, require re-authentication before proceeding with sensitive actions. I can't just run `sudo` without re-authenticating myself; and my ssh agent will reauthenticate me as well. See, e.g., https://developer.1password.com/docs/ssh/agent/security

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#70

Post by baobun »

chuckadams wrote: post #35
madeofpalk wrote: post #32

Earlier quoted context omitted.

As mentioned in the article, good NPM package managers just do this now. pnpm does it by default, yarn can be configured. Not sure about npm itself.

Got any pointers on how to configure this for yarn? I'm not turning anything up in the yarn documentation or in my random google searches. npm still seems to be debating whether they even want to do it. One of many reasons I ditched npm for yarn years ago (though the initial impetus was npm's confused and constantly changing behaviors around peer dependencies)

Yarn is unfortunately a dead-end security-wise under current maintainership.

If you are still on yarn v1 I suggest being consistent with '--ignore-scripts --frozen-lockfile' and run any necessary lifecycle scripts for dependencies yourself. There is @lavamoat/allow-scripts to manage this if your project warrants it.

If you are on newer yarn versions I strongly encourage to migrate off to either pnpm or npm.

Post reply on HN