Over 100k Infected Repos Found on GitHub • Hacker News Board

Re: Over 100k Infected Repos Found on GitHub

#121

Post by jonnat » Thu, Feb 29, 2024, 2:52 PM UTC

simonw wrote: post #89

Cupprum wrote: post #74

Earlier quoted context omitted.

Getting the actual number is probably very hard. These are the infected repos the OP found during their research.

For public repos you can get an approximate number by querying various public datasets. SELECT uniqHLL12(repo_name) FROM github_events; Against https://play.clickhouse.com/play?user=play#U0VMRUNUIHVuaXFIT... returns: 361648383

They probably mean that the actual number of malicious repos is probably very hard to get.

The article reaches the 100K number by searching for repos with patches with a particular string contained in this specific attack, so it's likely missing many malicious repos that use different methods of infection.

Re: Over 100k Infected Repos Found on GitHub

#122

Post by simonw » Thu, Feb 29, 2024, 3:00 PM UTC

jprete wrote: post #112

simonw wrote: post #82

Earlier quoted context omitted.

It's a risk. Similar to the risk that if you accept PRs from coworkers without reviewing them they might have copied and pasted in some vulnerable code from somewhere. Using LLMs means investing more effort in code review. I think that's a worthwhile trade.

This is not a worthwhile trade. The person sending the PR should not send LLM code that they can't vouch for, under the expectation that reviewers will find any vulnerabilities. That's just dumping the work onto the reviewers.

It's a worthwhile trade. I've been able to produce a substantial amount of reliable, working code with the assistance of LLMs over the past year - code I would have not been able to produce otherwise simply due to lack of time.

Why review code at all if you think your coworkers are infallible?

Re: Over 100k Infected Repos Found on GitHub

#123

Post by remram » Thu, Feb 29, 2024, 3:02 PM UTC

yencabulator wrote: post #107

justinclift wrote: post #88

Wonder if the whole curl + sudo shell script installer thing is going to come to an end any time soon? aka the whole "just run 'curl https://somesite/install.sh ' | sudo sh" to install our software Seems like it'd go very hand in hand with this infected stuff mentioned in the article.

Unfortunately `npm i` has the same power. `go get` is the only common dependency downloader I am currently aware of where hostile code doesn't run at install or build time. I think we need better tooling for working in sandboxes, to at least compartmentalize the explosion. ChromeOS's "virtual machines can open Wayland windows on the main desktop" trick is neat, but the code needed to do that was less than clean or re…

What's the point? You're going to ship that code to your users, or run it against your production database. If you think it might be malicious, protecting the dev laptop shouldn't be the priority.

Re: Over 100k Infected Repos Found on GitHub

#124

Post by keepamovin » Thu, Feb 29, 2024, 3:03 PM UTC

bunderbunder wrote: post #97

keepamovin wrote: post #71

This sucks. Supply chain is such an issue. Even tho we don't currently target any npm releases, I make use of socket.dev to monitor my project by creating an npm release for it. But my project BrowserBox (lightweight virtualized web browser) only uses ~800 dependencies including all descendents, with only 19 top-level deps (cool your heels non-JavaScript folks, this is comparatively lightweight for a full stack boing…

"only uses ~800 dependencies" slightly horrifies me. I was horrified to see how much time I started spending fussing with dependency hell after I moved from .NET to Java about 10 years back. And I am currently horrified by how much time I have to spend doing vulnerability updates and fussing with dependency hell in both Java and Python projects nowadays. I think maybe the reason I didn't have this problem to nearly t…

Totally, man. I like all this. I appreciate you sharing these war stories. :)

Re: Over 100k Infected Repos Found on GitHub

#125

Post by [deleted] » Thu, Feb 29, 2024, 3:05 PM UTC

zoover2020 wrote: post #102

efortis wrote: post #55

Earlier quoted context omitted.

Correct, the above mitigation is only for malware on the dev laptops and build servers. IOW, it doesn't prevent injecting the malware on your program when compiling it.

Shouldn't build servers have limited or zero network connectivity in the first place?

[deleted]

Re: Over 100k Infected Repos Found on GitHub

#126

Post by cogman10 » Thu, Feb 29, 2024, 3:06 PM UTC

mgii wrote: post #115

hollerith wrote: post #56

Earlier quoted context omitted.

Although finding over 100k infected repos is not good, it does not mean github is failing because the kind of programmer who would include an infected repo can find many other ways to create an insecure product if there weren't infected repos on github.

To be fair, the kind of programmer who would include an infected repo is almost everyone. Many infected repos have no indicators except for username to help you notice without a careful examination, especially in niche repos. When you have to move fast, it's natural to make such mistakes.

Further, transitive dependencies are a real risk. If A depends on B depends on C depends on D depends on E depends on F, and F is compromised which the author of E does not catch, everyone depending on any of the deps in the chain are at risk.

It's why the JavaScript ecosystem of micro packages is absolutely insane. If someone infected isEven, they'd have a blast radius of 90% of JavaScript devs.

It's much like having a single password protecting everything. JavaScript has way too many of these high value packages that find their way into every modern JavaScript project.

Re: Over 100k Infected Repos Found on GitHub

#127

Post by remram » Thu, Feb 29, 2024, 3:07 PM UTC

pas wrote: post #51

efortis wrote: post #48

For NPMs you can mitigate executing malware with `--ignore-scripts` https://blog.uirig.com/getting-rid-of-npm-scripts

instead the downloaded malicious code runs in prod. maybe, if you are lucky, it does something strange in CI and you can catch it. the only real solution is a reputation system (like https://github.com/crev-dev/cargo-crev ), which of course is unfortunately barely used

Any equivalent for non-Rust projects? I see git-crev is abandoned...

Re: Over 100k Infected Repos Found on GitHub

#128

Post by dartos » Thu, Feb 29, 2024, 3:12 PM UTC

pjc50 wrote: post #73

dspillett wrote: post #65

As well as this being our regular reminder to be careful what you pull from public repositories and other sources, and to verify your dependency trees, it raises another question: If malware is massively prolific in public repos, how much does this affect LLMs and other automation tools that are trained using the contents of such resources? What are the chances that we'll see copilot & friends occasionally emit malwa…

I'd expect it to accidentally invent vulnerabilities of its own as well as pasting existing ones from the input set. AI provides no guarantees at all about correctness.

You can use grammars to guarantee correct syntax, at least.

Re: Over 100k Infected Repos Found on GitHub

#129

Post by dartos » Thu, Feb 29, 2024, 3:13 PM UTC

simonw wrote: post #122

jprete wrote: post #112

Earlier quoted context omitted.

This is not a worthwhile trade. The person sending the PR should not send LLM code that they can't vouch for, under the expectation that reviewers will find any vulnerabilities. That's just dumping the work onto the reviewers.

It's a worthwhile trade. I've been able to produce a substantial amount of reliable, working code with the assistance of LLMs over the past year - code I would have not been able to produce otherwise simply due to lack of time. Why review code at all if you think your coworkers are infallible?

I’m fine with LLM code as long as someone actually understands what it’s doing.

Re: Over 100k Infected Repos Found on GitHub

#130

Post by lopkeny12ko » Thu, Feb 29, 2024, 3:15 PM UTC

sigmoid10 wrote: post #2

I've noticed these too by randomly stumbling over similar repos. I usually don't run code from random repos, but now I have reached a point where I spin up a sandbox vm even when I trust the repo and the owners. If you are a dev today, you should probably have at least thee firmly separated environments for work, hobby and personal stuff.

I do this now today too, but not even for potentially malicious software. Some projects that are not inherently malicious are just written poorly or stupidly by design. Just the other day I ran a program that, before I even requested it to do anything, appended 3 lines to my ~/.bashrc. I didn't even notice until days later. I can't fathom why any developer thinks this is a good idea, and is exactly the kind of thing that makes me sandbox every foreign piece of code I run now.