Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Antigravity exfiltrates data via indirect prompt injection attack

promptarmor.com

Post reply on HN
Print view View original

151–160 of 227 posts

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#151

Post by jermaustin1 »

lazide wrote: post #53
jermaustin1 wrote: post #13

Earlier quoted context omitted.

Because the article shows it isn't Gemini that is the issue, it is the tool calling. When Gemini can't get to a file (because it is blocked by .gitignore), it then uses cat to read the contents. I've watched this with GPT-OSS as well. If the tool blocks something, it will try other ways until it gets it. The LLM "hacks" you.

And… that isn’t the LLM’s fault/responsibility?

How can an LLM be at fault for something? It is a text prediction engine. WE are giving them access to tools.

Do we blame the saw for cutting off our finger? Do we blame the gun for shooting ourselves in the foot? Do we blame the tiger for attacking the magician?

The answer to all of those things is: no. We don't blame the thing doing what it is meant to be doing no matter what we put in front of it.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#153

Post by lazide »

jermaustin1 wrote: post #151
lazide wrote: post #53

Earlier quoted context omitted.

And… that isn’t the LLM’s fault/responsibility?

How can an LLM be at fault for something? It is a text prediction engine. WE are giving them access to tools. Do we blame the saw for cutting off our finger? Do we blame the gun for shooting ourselves in the foot? Do we blame the tiger for attacking the magician? The answer to all of those things is: no. We don't blame the thing doing what it is meant to be doing no matter what we put in front of it.

It was not meant to give access like this. That is the point.

If a gun randomly goes off and shoots someone without someone pulling the trigger, or a saw starts up when it’s not supposed to, or a car’s brakes fail because they were made wrong - companies do get sued all the time.

Because those things are defective.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#154

Post by jacquesm »

mudkipdev wrote: post #150
simonw wrote: post #143

Earlier quoted context omitted.

This isn't a bug in the LLMs. It's a bug in the software that uses those LLMs. An LLM on its own can't execute code. An LLM harness like Antigravity adds that ability, and if it does it carelessly that becomes a security vulnerability.

No matter how many prompt changes you make it won't be possible to fix this.

So, what's your conclusion from that bit of wisdom?

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#155

Post by jacquesm »

cowpig wrote: post #26
ArcHound wrote: post #19

Who would have thought that having access to the whole system can be used to bypass some artificial check. There are tools for that, sandboxing, chroots, etc... but that requires engineering and it slows GTM, so it's a no-go. No, local models won't help you here, unless you block them from the internet or setup a firewall for outbound traffic. EDIT: they did, but left a site that enables arbitrary redirects in the de…

> No, local models won't help you here, unless you block them from the internet or setup a firewall for outbound traffic. This is the only way. There has to be a firewall between a model and the internet. Tools which hit both language models and the broader internet cannot have access to anything remotely sensitive. I don't think you can get around this fact.

And here we have google pushing their Gemini offering inside the Google cloud environment (docs, files, gmail etc) at every turn. What could possibly go wrong?

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#156

Post by Nathanba »

paxys wrote: post #50

I'm not quite convinced. You're telling the agent "implement what it says on " and the blog is malicious and exfiltrates data. So Gemini is simply following your instructions. It is more or less the same as running "npm install " on your own. Ultimately, AI or not, you are the one responsible for validating dependencies and putting appropriate safeguards in place.

right but this product (agentic AI) is explicitly sold as being able to run on its own. So while I agree that these problems are kind of inherent in AIs... these companies are trying to sell it anyway even though they know that it is going to be a big problem.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#157

Post by blazespin »

wingmanjd wrote: post #46

I really liked Simon's Willison's [1] and Meta's [2] approach using the "Rule of Two". You can have no more than 2 of the following: - A) Process untrustworthy input - B) Have access to private data - C) Be able to change external state or communicate externally. It's not bullet-proof, but it has helped communicate to my management that these tools have inherent risk when they hit all three categories above (and any…

You can't process untrustworthy data, period. There are so many things that can go wrong with that.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#158

Post by yakbarber »

blazespin wrote: post #157
wingmanjd wrote: post #46

I really liked Simon's Willison's [1] and Meta's [2] approach using the "Rule of Two". You can have no more than 2 of the following: - A) Process untrustworthy input - B) Have access to private data - C) Be able to change external state or communicate externally. It's not bullet-proof, but it has helped communicate to my management that these tools have inherent risk when they hit all three categories above (and any…

You can't process untrustworthy data, period. There are so many things that can go wrong with that.

that's basically saying "you can't process user input". sure you can take that line, but users wont find your product to be very useful

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#159

Post by zahlman »

bitbasher wrote: post #99
ArcHound wrote: post #19

Who would have thought that having access to the whole system can be used to bypass some artificial check. There are tools for that, sandboxing, chroots, etc... but that requires engineering and it slows GTM, so it's a no-go. No, local models won't help you here, unless you block them from the internet or setup a firewall for outbound traffic. EDIT: they did, but left a site that enables arbitrary redirects in the de…

> Who would have thought that having access to the whole system can be used to bypass some artificial check. You know, years ago there was a vulnerability through vim's mode lines where you could execute pretty random code. Basically, if someone opened the file you could own them. We never really learn do we? CVE-2002-1377 CVE-2005-2368 CVE-2007-2438 CVE-2016-1248 CVE-2019-12735 Do we get a CVE for Antigravity too?

> a vulnerability through vim's mode lines where you could execute pretty random code. Basically, if someone opened the file you could own them.

... Why would Vim be treating the file contents as if they were user input?

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#160

Post by helsinki »

wingmanjd wrote: post #46

I really liked Simon's Willison's [1] and Meta's [2] approach using the "Rule of Two". You can have no more than 2 of the following: - A) Process untrustworthy input - B) Have access to private data - C) Be able to change external state or communicate externally. It's not bullet-proof, but it has helped communicate to my management that these tools have inherent risk when they hit all three categories above (and any…

Yeah, makes perfect sense, but you really lose a lot.
Post reply on HN