Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos

noma.security

Post reply on HN
Print view View original

51–60 of 228 posts

Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos

#51

Post by dainiusse »

zero_k wrote: post #49

Nobody at GitHub expected this? Their feature develoment&release processes must be garbage/non-existent/not followed. This potential security issue should have been flagged when the new feature was thought up, security should have been part of the process of implementing the feature giving continuous feedback, and it should have been tested for before release of the feature. That's how modern security teams work in l…

You know how it works. There probably were people who didn't want that, but then there is push from business, deadlines, etc.

Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos

#52

Post by rileymat2 »

LoganDark wrote: post #33
valleyer wrote: post #31

Earlier quoted context omitted.

Your second paragraph directly contradicts the first.

Since you cannot fix information leakage from LLMs, you must remove the information so that it cannot be leaked. There is no contradiction there.

There is a major contradiction depending on the definition of “support staff” and the role of the llm in the system which may need access to sensitive data or systems to perform its functions.

Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos

#53

Post by rrr_oh_man »

mcv wrote: post #42
fwlr wrote: post #37

“Prompt injection attacks have become, to agentic AI, what SQL injections were to web applications: a systematic, category-wide vulnerability class that requires the same systematic strategies and defenses.” ??? Isn’t prompt injection far more fatal to LLMs than SQL injection is to SQL databases? Like, the problem of SQL injection was that user input was forming part of the instruction string given to the SQL engine,…

Exactly. SQL injection was caused by treating user input as part of the instruction instead of as the pure data that it was intended as. Separating those two fixed it. Prompt injection is unavoidable because the user input is intended as instruction.

What do you mean by "was" and "fixed it"? It is still very much an issue and remains in the OWASP Top 10.

https://owasp.org/Top10/2025/A05_2025-Injection/

Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos

#54

Post by amelius »

fwlr wrote: post #37

“Prompt injection attacks have become, to agentic AI, what SQL injections were to web applications: a systematic, category-wide vulnerability class that requires the same systematic strategies and defenses.” ??? Isn’t prompt injection far more fatal to LLMs than SQL injection is to SQL databases? Like, the problem of SQL injection was that user input was forming part of the instruction string given to the SQL engine,…

"We can't fix it, therefore we just keep using it."

Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos

#55

Post by salviati »

rrr_oh_man wrote: post #53
mcv wrote: post #42

Earlier quoted context omitted.

Exactly. SQL injection was caused by treating user input as part of the instruction instead of as the pure data that it was intended as. Separating those two fixed it. Prompt injection is unavoidable because the user input is intended as instruction.

What do you mean by "was" and "fixed it" ? It is still very much an issue and remains in the OWASP Top 10. https://owasp.org/Top10/2025/A05_2025-Injection/

You can write your code so SQL injections are not possible.

You can't do the same with prompt injections.

Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos

#56

Post by formerly_proven »

rrr_oh_man wrote: post #53
mcv wrote: post #42

Earlier quoted context omitted.

Exactly. SQL injection was caused by treating user input as part of the instruction instead of as the pure data that it was intended as. Separating those two fixed it. Prompt injection is unavoidable because the user input is intended as instruction.

What do you mean by "was" and "fixed it" ? It is still very much an issue and remains in the OWASP Top 10. https://owasp.org/Top10/2025/A05_2025-Injection/

sqli is easily and fully mitigated and has generally been a non-issue for any half-serious project, especially if you use any kind of SAST. Your link actually subsumes any type of injection, not just sqli. Some of them are marginally harder to fix than sqli, most aren't.

In contrast, we don't know how to solve prompt injection.

Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos

#58

Post by reactordev »

AgentMatt wrote: post #47
jakewins wrote: post #28

How is this a Github vulnerability? The researchers are the ones that grant the agent access to private repos and then ask it to answer questions in public repos.. of course this allows extracting private information? This is like setting up a normal CI job with access to secrets and running it on public PRs. If you configure GitHub to allow public code or LLM instructions to run in contexts that have access to sensi…

Agreed. It seems a core issue underlying these prompt injection attacks is a failure to properly scope the agent's permissions. In this case, depending on what exactly the agent is supposed to actually do, this might be defining a separate workflow agent per repo, or a workflow agent with broader repo access but configured to only be triggered by users on an allow list (still compatible with developing in the open, s…

The fallacy here is expecting an agent that has access to ALL your repos to respect the singular repo it’s in. It won’t. If it has access to all your repos and you ask it about a private repo you aren’t in - it will definitely go look at that private repo. This is like giving your dog a bone and then being surprised when he buries it in the backyard.

Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos

#59

Post by adamddev1 »

alex_suzuki wrote: post #50
adamddev1 wrote: post #13

Earlier quoted context omitted.

I'm done. Moving to Forgejo. It's wonderful and everything works better. Seriously like everything is instant when you click around, and CI with a runner works beautifully. (The documentation for setting up the runner could be a tad clearer but otherwise everything was so painless.)

Self-hosted, or are you using something managed? I’ve held off switching from Gitlab for now as everything is setup and runs ok, but they’re pushing their AI hard into every corner. Not a lot of good managed options around (yet), especially in Europe. Codey ( https://www.codey.ch/ ) is pretty expensive and doesn’t offer runners out of the box.

Self-hosted. It runs great on a tiny VPS with other services. But I did have to get a cheaper Hetzner server (5 Euros-ish for 4GB RAM) to run the runner.

Forgejo feels like a refreshing blast from the past. No intrusive AI cramming. The Web Interface is snappy and responsive, not waiting for constant loaders and spinners. It takes almost no resources to run.

Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos

#60

Post by philipp-gayret »

hobofan wrote: post #40
jakewins wrote: post #28

How is this a Github vulnerability? The researchers are the ones that grant the agent access to private repos and then ask it to answer questions in public repos.. of course this allows extracting private information? This is like setting up a normal CI job with access to secrets and running it on public PRs. If you configure GitHub to allow public code or LLM instructions to run in contexts that have access to sensi…

> If you configure GitHub to allow public code or LLM instructions to run in contexts that have access to sensitive things, they will leak; that’s not GitHub’s fault, it’s yours. Is there a way to segment access per agentic workflow, so that you can have both habe an agentic workflow that has access to sensitive data and one that has only access to public data? Is the default to set the scope to only the current repo…

> Is there a way to segment access per agentic workflow, so that you can have both habe an agentic workflow that has access to sensitive data and one that has only access to public data? Is the default to set the scope to only the current repository?

If the author had used the native secrets.GITHUB_TOKEN then yes.

> Does Github appropriately inform about the risk of combining an agentic workflow with access to private repository data?

Not really, but also this highlights a broader issue: GitHub introduced fine-grained access tokens quite a while ago to prevent these situations. However, fine-grained access tokens don't work for a fair segment of the GitHub API for whatever reason. So often you have to use a personal access token to create a GitHub integration, and these have extremely broad permissions. Having said that, that is still the author's choice.

Post reply on HN