Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Remote Prompt Injection in Gitlab Duo Leads to Source Code Theft

legitsecurity.com

Post reply on HN
Print view View original

51–57 of 57 posts

Re: Remote Prompt Injection in Gitlab Duo Leads to Source Code Theft

#51

Post by cutemonster »

Kholin wrote: post #50

If Duo were a web application, then would properly setting the Content Security Policy (CSP) in the page response headers be enough to prevent these kinds of issues? https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/CSP

To stop exfiltration via images? Yes seems so? If you configure img-src:

  The first directive, default-src, tells the browser to load only resources that are same-origin with the document, unless other more specific directives set a different policy for other resource types.

  The second, img-src, tells the browser to load images that are same-origin or that are served from example.com.
But that wouldn't stop the AI from writing dangerous instructions in plain text to the human

Re: Remote Prompt Injection in Gitlab Duo Leads to Source Code Theft

#52

Post by zwnow »

thaumasiotes wrote: post #40
zwnow wrote: post #25

Earlier quoted context omitted.

For tricking individuals your first got to contact them somehow. To trick an LLM you can just spam prompts.

You email them. It's called phishing.

Employees usually know to not click on random shit they get sent. Most mails alrdy get filtered before they even reach the employee. Good luck actually achieving something with phishing mails.

Re: Remote Prompt Injection in Gitlab Duo Leads to Source Code Theft

#53

Post by thaumasiotes »

zwnow wrote: post #52
thaumasiotes wrote: post #40

Earlier quoted context omitted.

You email them. It's called phishing.

Employees usually know to not click on random shit they get sent. Most mails alrdy get filtered before they even reach the employee. Good luck actually achieving something with phishing mails.

When I was at NCC Group, we had a policy about phishing in penetration tests.

The policy was "we'll do it if the customer asks for it, but we don't recommend it, because the success rate is 100%".

Re: Remote Prompt Injection in Gitlab Duo Leads to Source Code Theft

#54

Post by serf »

margalabargala wrote: post #44
tough wrote: post #36

Earlier quoted context omitted.

an agent does rm -rf / i think i saw it do it or try it and my computer shut down and restarted (mac) maybe it just deleted the project lol these llms are really bad at keeping track of the real world, so they might think they're on the project folder but had just navigated back with cd to the user ~ root and so shit happens. Honestly one should run only these on controlled env's like VM's or Docker. but YOLO amirite

That people allow these agents to just run arbitrary commands against their primary install is wild . Part of this is the tool's fault. Anything like that should be done in a chroot. Anything less is basically "twitch plays terminal" on your machine.

a large part of the benefit to an agentic ai is that it can coordinate tests that it automatically wrote on an existing code base, a lot of time the only way to get decent answers out of something like that is to let it run as bare metal as it can. I run cursor and the accompanying agents in a snapshot'd VM for this purpose. It's not much different than what you suggest, but the layer of abstraction is far enough for admin-privileged app testing, an unfortunate reality for certain personal projects.

I haven't had a cursor install nuke itself yet, but I have had one fiddling in a parent folder it shouldn't have been able to with workspace protection on..

Re: Remote Prompt Injection in Gitlab Duo Leads to Source Code Theft

#55

Post by bluefirebrand »

thaumasiotes wrote: post #53
zwnow wrote: post #52

Earlier quoted context omitted.

Employees usually know to not click on random shit they get sent. Most mails alrdy get filtered before they even reach the employee. Good luck actually achieving something with phishing mails.

When I was at NCC Group, we had a policy about phishing in penetration tests. The policy was "we'll do it if the customer asks for it, but we don't recommend it, because the success rate is 100%".

How can you ever get that lower than 100% if you don't do the test to identify which employees need to be trained / monitored because they fall for phishing?

Re: Remote Prompt Injection in Gitlab Duo Leads to Source Code Theft

#56

Post by tough »

TechDebtDevin wrote: post #49
tough wrote: post #36

Earlier quoted context omitted.

an agent does rm -rf / i think i saw it do it or try it and my computer shut down and restarted (mac) maybe it just deleted the project lol these llms are really bad at keeping track of the real world, so they might think they're on the project folder but had just navigated back with cd to the user ~ root and so shit happens. Honestly one should run only these on controlled env's like VM's or Docker. but YOLO amirite

This is what happened. I was testing claude 4 and asked it to create a simple 1K LOC fyne android app. I have my repos stored outside of my linux user so the work it created was preserved. It essentially created a bash file that cd ~ && rm -rf / . All settings reset and documents/downloads disappeared lmfao. I don't ever really use my OS as primary storage, and any config or file of importance is backed up twice so i…

if you think deeply about it, its one kind of harakiri as an AI to remove the whole system you're operating on.

Yeah Claude 4 can go too far some times

Re: Remote Prompt Injection in Gitlab Duo Leads to Source Code Theft

#57

Post by sunnybeetroot »

TechDebtDevin wrote: post #28
cedws wrote: post #4

Until prompt injection is fixed, if it is ever, I am not plugging LLMs into anything. MCPs, IDEs, agents, forget it. I will stick with a simple prompt box when I have a question and do whatever with its output by hand after reading it.

Cursor deleted my entire Linux user and soft reset my OS, so I dont blame you.

Cursor by default asks to execute commands, sounds like you had auto run commands on…
Post reply on HN