Live data from Hacker News

Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

trigger.dev

161–170 of 191 posts

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#161
post #50

Earlier quoted context omitted.

I don’t think that’s considered secure enough, see the other answers and the push for passkeys. I mean, if passphrases were good for anything you’d directly use them for the ssh connection? :)

Passphrases, when strong enough, are fine when they are not traversing a medium that can be observed by a third party. They're not recommended for authenticating a secure connection over a network, but they’re fine for unlocking a much longer secret that cannot be cracked via guessing, rainbow tables, or other well known means. Hell, most people unlock their phones with a 4 digit passcode, and their computers with a…

> when they are not traversing a medium that can be observed by a third party

Isn't that why all those security experts are pushing for SSL everywhere and 30 second certificate expiration? To make the medium unobservable by a third party?

If you believe them, passphrases should be okay over fiber you don't control too.

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#162

Very offtopic but this caught my eye: > Total repos cloned: 669 How big is this company? All the numbers I can find online suggest well below 100 people, and yet they have over 600 repos? Is that normal?

We have a ratio of roughly 7:1 (repos to engineers). It was probably closer to 12:1 at some point.

* Spikes/Demo project

* Smaller projects that might have gone live, but have since been migrated elsewhere

* Core services

* Forks of certain supply chain dependencies that we've made improvements to.

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#163

Earlier quoted context omitted.

Passphrases, when strong enough, are fine when they are not traversing a medium that can be observed by a third party. They're not recommended for authenticating a secure connection over a network, but they’re fine for unlocking a much longer secret that cannot be cracked via guessing, rainbow tables, or other well known means. Hell, most people unlock their phones with a 4 digit passcode, and their computers with a…

> when they are not traversing a medium that can be observed by a third party Isn't that why all those security experts are pushing for SSL everywhere and 30 second certificate expiration? To make the medium unobservable by a third party? If you believe them, passphrases should be okay over fiber you don't control too.

One thing I forgot to mention is what the trust relationship looks like. Passphrases used for authentication are known by both parties and could be leaked by the other side or stolen from them, while private keys remain only available to you. With public key authentication, the other party only has your public key, which is freely shareable.

And yes, we all know that 2FA, passkeys, etc. are all better than passphrases, and that layer 3 wire encryption is important.

I’m merely responding to your blanket assertion that passphrases aren’t “secure enough,” but sometimes they are.

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#164
post #48
post #3

I have been thinking about this. How do I make my git setup on my laptop secure? Currently, I have my ssh key on the laptop, so if I want to push, I just use git push. And I have admin credentials for the org. How do I make it more secure?

There is no defense against a compromised laptop. You should prevent this at all cost. You can make it a bit more challenging for the attacker by using secure enclaves (like TPM or Yubikey), enforce signed commits, etc. but if someone compromised your machine, they can do whatever you can. Enforcing signing off on commits by multiple people is probably your only bet. But if you have admin creds, an attacker can turn…

This is absolutely not true.

A compromised laptop should always be treated as a fully compromised. However, you can take steps that drastically reduce the likelihood of bad things happening before you can react (e.g. disable accounts/rotate keys).

Further, you can take actions that inherently limit the ability for a compromise to actually cause impact. Not needing to actually store certain things on the machine is a great start.

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#165

Earlier quoted context omitted.

It has nothing to do with interpreters or JIT, it has nothing to do with npm at all. All package managers have the insane security model of "arbitrary code execution with no constraints".

It just so happens that all of those languages share the worst design points, such as the need for a package manager at all and the classic "eval and equivalents run arbitrary code". >All package managers have the insane security model of "arbitrary code execution with no constraints". Not all of them, just the most popular ones for these highly sophisticated, well thought-out bunch of absolute languages.

What language does not have a popular package manager that provides code execution?

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#166

Earlier quoted context omitted.

It has nothing to do with interpreters or JIT, it has nothing to do with npm at all. All package managers have the insane security model of "arbitrary code execution with no constraints".

I tend to agree but think npms post install hook is a degree worse. Triggering during install, silently because npm didn't like someone using the feature to ask for donations, is worse than requiring you to load and run the package code.

Which package managers don't contain an equivalent feature for running code as part of the install process?

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#167
post #125

Earlier quoted context omitted.

They can't explain, it's just victim blaming. The market currently doesn’t have a proper solution to this. Everyone works with these package managers, I bet the commenter also has installed pip or npm packages without reading its full code, it just feels cool to tell other people they are dumb and it's their own fault for not reading all the code beforehand or for using a package manager, when every single person doe…

It isn't victim blaming. People like you make it impossible to avoid attacks like these because you have no appetite for a better security model. I run npm under bubblewrap because npm has a culture of high risk; of using too many dependencies from untrusted authors. But being scrupulous and responsible is a cost I pay with my time and attention. But it is important because if I run some untrusted code and am comprom…

What no appetite? I just don't like your solution. The industry needs an answer to this problem stat, and it can't be "just read the code before".

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#168
post #113
post #27

Earlier quoted context omitted.

There are logs for accessing aws resources and if you don't see the access before you revoke it then the data is safe

Because an attacker would never cover their tracks...

Indeed, being able to trust your audit logs is imperative.

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#169
post #143

Earlier quoted context omitted.

Completely normal yes. Repos are cattle not pets.

> Repos are cattle not pets. What do you mean by this?

A core SRE principle is that "machines/servers are cattle, not pets". They shouldn't be special or bespoke in a way that makes replacement painful or difficult.

Re: Shai-Hulud compromised a dev machine and raided GitHub org access: a post-mortem

#170

Earlier quoted context omitted.

Or use a FIDO token to protect your SSH key, which becomes useless without the hardware token. https://wiki.archlinux.org/title/SSH_keys#FIDO/U2F That's what I do. For those of us too lazy to read the article, tl;dr: ssh-keygen -t ed25519-sk or, if your FIDO token doesn't support edwards curves: ssh-keygen -t ecdsa-sk tap the token when ssh asks for it, done. Use the ssh key as usual. OpenSSH will ask you to tap the…

Except that an attacker can modify the ssh config to enable session multiplexing with a long timeout and then piggy-back off that connection, right?

Looks like on the server side this can be mitigated somewhat by the MaxStartups¹ setting for OpenSSH or equivalent behavior for other services that support SSH auth (e.g., Git forges like GitHub):

  MaxStartups
               Specifies the maximum number of concurrent unauthenticated
               connections to the SSH daemon.  Additional connections
               will be dropped until authentication succeeds or the
               LoginGraceTime expires for a connection.  The default is
               10:30:100.

               Alternatively, random early drop can be enabled by
               specifying the three colon separated values
               start:rate:full (e.g. "10:30:60").  sshd(8) will refuse
               connection attempts with a probability of rate/100 (30%)
               if there are currently start (10) unauthenticated
               connections.  The probability increases linearly and all
               connection attempts are refused if the number of
               unauthenticated connections reaches full (60).
So it looks like it's possible to support ControlMaster while still somewhat hampering mass-cloning thousands of repos via SSH key without reauthenticating.

Admittedly I'd put this more in the category of making endpoint compromise easier to detect than that of actually preventing any particular theft of data or manipulation of systems. But it might still be worth doing! If it means only a few dozen or only a hundred repos get compromised before detection instead of a few thousand, that's a good thing.

Besides all that (or MaxSessions, as another user mentions), if an attacker compromises a developer laptop and can only open those connections as long as the developer is online, that's one thing. But a plaintext key that they can grab and reuse from their own box is obviously an even sweeter prize!

"The SSH key on my YubiKey is useless to attackers" is obviously the wrong way to think about this, but using a smartcard for SSH keys is still a way to avoid storing plaintext secrets. It's good hygiene.

--

https://www.man7.org/linux/man-pages/man5/sshd_config.5.html

Post reply on HN