GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos
1–10 of 228 posts
Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos
#2[deleted]
Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos
#31. The issue is already solved.
2. Or issue is not solved yet by GitHub, and meanwhile bad actors gonna try vulnerability on repos. Due to number of repos there is non-zero probability. But as with scams almost nobody’s going to admit the leakage.
Anything else?
Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos
#4Who thought having a LLM with access to private information, with public access to ask it questions, would ever be a secure process?
Look I like interacting with these tools as much as the next guy, but I'm certainly not going to trust them with access to information and then allow anyone to send them prompts.
Edit/further thoughts: So (assumable as they said this is disclosed with github's knowledge) this has been patched. But how many different word combinations will it take to find another way to have this occur?
Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos
#5Who thought having a LLM with access to private information, with public access to ask it questions, would ever be a secure process? Look I like interacting with these tools as much as the next guy, but I'm certainly not going to trust them with access to information and then allow anyone to send them prompts. Edit/further thoughts: So (assumable as they said this is disclosed with github's knowledge) this has been p…
Yea agreed. LLM guardrails are either just written prompts as in "Please do not bad stuff :(" or other LLMs verifying that the first LLM didn't so some bs. Both of wich methods do not work sufficiently as time shows again and again.
Funnily enough, nobody expects quality software anymore and errors became tolerable. So thats a win (for someone like me that lost all passion for the industry).
Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos
#6Who thought having a LLM with access to private information, with public access to ask it questions, would ever be a secure process? Look I like interacting with these tools as much as the next guy, but I'm certainly not going to trust them with access to information and then allow anyone to send them prompts. Edit/further thoughts: So (assumable as they said this is disclosed with github's knowledge) this has been p…
It must be something to do with Microsoft being the owner now of GitHub
Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos
#7Who thought having a LLM with access to private information, with public access to ask it questions, would ever be a secure process? Look I like interacting with these tools as much as the next guy, but I'm certainly not going to trust them with access to information and then allow anyone to send them prompts. Edit/further thoughts: So (assumable as they said this is disclosed with github's knowledge) this has been p…
It must be something to do with Microsoft being the owner now of GitHub
You know what? I had honestly forgotten about that xD. /thread
Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos
#8Who thought having a LLM with access to private information, with public access to ask it questions, would ever be a secure process? Look I like interacting with these tools as much as the next guy, but I'm certainly not going to trust them with access to information and then allow anyone to send them prompts. Edit/further thoughts: So (assumable as they said this is disclosed with github's knowledge) this has been p…
Yea agreed. LLM guardrails are either just written prompts as in "Please do not bad stuff :(" or other LLMs verifying that the first LLM didn't so some bs. Both of wich methods do not work sufficiently as time shows again and again. Funnily enough, nobody expects quality software anymore and errors became tolerable. So thats a win (for someone like me that lost all passion for the industry).
Agree with your assessment of guardrails. They barely work on the best days. We need to flip the idea of “agent” on its head. The agent here is an agent of the user interfacing with GitHub. Not an agent of GitHub interfacing with the user. Prompts and guardrails cannot keep the agent loyal to the company. Stop giving these things any permissions the user doesn’t have, and recognize them for what they are: a different UI than web forms, but still the same security model.
Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos
#9Who thought having a LLM with access to private information, with public access to ask it questions, would ever be a secure process? Look I like interacting with these tools as much as the next guy, but I'm certainly not going to trust them with access to information and then allow anyone to send them prompts. Edit/further thoughts: So (assumable as they said this is disclosed with github's knowledge) this has been p…
Yea agreed. LLM guardrails are either just written prompts as in "Please do not bad stuff :(" or other LLMs verifying that the first LLM didn't so some bs. Both of wich methods do not work sufficiently as time shows again and again. Funnily enough, nobody expects quality software anymore and errors became tolerable. So thats a win (for someone like me that lost all passion for the industry).
That last part is I think called negligence. And in some industries that becomes criminal negligence quite quickly.
Re: GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos
#10Who thought having a LLM with access to private information, with public access to ask it questions, would ever be a secure process? Look I like interacting with these tools as much as the next guy, but I'm certainly not going to trust them with access to information and then allow anyone to send them prompts. Edit/further thoughts: So (assumable as they said this is disclosed with github's knowledge) this has been p…
My Lethal Trifecta talk at the Bay Area AI Security Meetup - https://news.ycombinator.com/item?id=44846922 - August 2025 (115 comments)