Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos

noma.security

Post reply on HN
Print view View original

21–30 of 228 posts

Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos

#21

Post by marak830 »

toomuchtodo wrote: post #10
marak830 wrote: post #4

Who thought having a LLM with access to private information, with public access to ask it questions, would ever be a secure process? Look I like interacting with these tools as much as the next guy, but I'm certainly not going to trust them with access to information and then allow anyone to send them prompts. Edit/further thoughts: So (assumable as they said this is disclosed with github's knowledge) this has been p…

My Lethal Trifecta talk at the Bay Area AI Security Meetup - https://news.ycombinator.com/item?id=44846922 - August 2025 (115 comments) https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/

Good read thanks.

Also interesting to see who coined the term prompt injection.

Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos

#24

Post by zzril »

> In most agentic prompt injection attacks, the agent treats the wrong content as a trusted source of instructions and allows itself to be misdirected or misused. This happens when the system fails to maintain a strict trust boundary between system-level directives and untrusted user data.

How on earth is a probabilistic token predictor supposed to turn untrusted user input into trusted system-level directives? The strict trust boundary must be maintained on this side of the agent, not within it.

Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos

#26

Post by Gigachad »

jofzar wrote: post #11

> Responsible Disclosure GitLost was responsibly disclosed to GitHub. Vulnerability details are shared here with their knowledge. Why does this section not have when it was fixed or GitHub acknowledge/rejected this? Did they not fix this?

This isn’t a normal software bug, it’s not fixable in the same way you can’t fix regular support staff from being tricked.

The answer is you should not allow LLMs access to untrusted input and sensitive data at the same time.

Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos

#28

Post by jakewins »

How is this a Github vulnerability? The researchers are the ones that grant the agent access to private repos and then ask it to answer questions in public repos.. of course this allows extracting private information?

This is like setting up a normal CI job with access to secrets and running it on public PRs. If you configure GitHub to allow public code or LLM instructions to run in contexts that have access to sensitive things, they will leak; that’s not GitHub’s fault, it’s yours.

Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos

#29

Post by stingraycharles »

jakewins wrote: post #28

How is this a Github vulnerability? The researchers are the ones that grant the agent access to private repos and then ask it to answer questions in public repos.. of course this allows extracting private information? This is like setting up a normal CI job with access to secrets and running it on public PRs. If you configure GitHub to allow public code or LLM instructions to run in contexts that have access to sensi…

"How is this a Github vulnerability? The researchers are the ones that grant the agent access to private repos and then ask it to answer questions in public repos.. of course this allows extracting private information?"

I think the assumption is that the permissions are scoped to the repository you're currently asking questions on, rather than your private repositories as well.

I can see arguments for both sides.

Post reply on HN