Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Antigravity exfiltrates data via indirect prompt injection attack

promptarmor.com

Post reply on HN
Print view View original

81–90 of 227 posts

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#81

Post by pmontra »

mkagenius wrote: post #22
NitpickLawyer wrote: post #16

Earlier quoted context omitted.

Because it misses the point. The problem is not the model being in a cloud. The problem is that as soon as "untrusted inputs" (i.e. web content) touch your LLM context, you are vulnerable to data exfil. Running the model locally has nothing to do with avoiding this. Nor does "running code in a sandbox", as long as that sandbox can hit http / dns / whatever. The main problem is that LLMs share both "control" and "data…

Sorry, I didn't elaborate. But "completely local" meant not doing any network calls unless specifically approved. When llm calls are completely local you just need to monitor a few explicit network calls to be sure.

In a realistic and useful scenario, how would you approve or deny network calls made by a LLM?

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#82

Post by buu700 »

IshKebab wrote: post #74
jsmith99 wrote: post #37

There's nothing specific to Gemini and Antigravity here. This is an issue for all agent coding tools with cli access. Personally I'm hesitant to allow mine (I use Cline personally) access to a web search MCP and I tend to give it only relatively trustworthy URLs.

I do think they deserve some of the blame for encouraging you to allow all commands automatically by default.

YOLO-mode agents should be in a dedicated VM at minimum, if not a dedicated physical machine with a strict firewall. They should be treated as presumed malware that just happens to do something useful as a side effect.

Vendors should really be encouraging this and providing tooling to facilitate it. There should be flashing red warnings in any agentic IDE/CLI whenever the user wants to use YOLO mode without a remote agent runner configured, and they should ideally even automate the process of installing and setting up the agent runner VM to connect to.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#83

Post by DrSusanCalvin »

kleiba wrote: post #59
alzoid wrote: post #23

Earlier quoted context omitted.

I had this issue today. Gemini CLI would not read files from my directory called .stuff/ because it was in .gitignore. It then suggested running a command to read the file ....

The AI needs to be taught basic ethical behavior: just because you can do something that you're forbidden to do, doesn't mean you should do it.

Unfortunately yes, teaching AI the entirety of human ethics is the only foolproof solution. That's not easy though. For example, what about the case where a script is not executable, would it then be unethical for the AI to suggest running chmod +x? It's probably pretty difficult to "teach" a language model the ethical difference between that and running cat .env

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#84

Post by btown »

wingmanjd wrote: post #46

I really liked Simon's Willison's [1] and Meta's [2] approach using the "Rule of Two". You can have no more than 2 of the following: - A) Process untrustworthy input - B) Have access to private data - C) Be able to change external state or communicate externally. It's not bullet-proof, but it has helped communicate to my management that these tools have inherent risk when they hit all three categories above (and any…

It's really vital to also point out that (C) doesn't just mean agentically communicate externally - it extends to any situation where any of your users can even access the output of a chat or other generated text.

You might say "well, I'm running the output through a watchdog LLM before displaying to the user, and that watchdog doesn't have private data access and checks for anything nefarious."

But the problem is that the moment someone figures out how to prompt-inject a quine-like thing into a private-data-accessing system, such that it outputs another prompt injection, now you've got both (A) and (B) in your system as a whole.

Depending on your problem domain, you can mitigate this: if you're doing a classification problem and validate your outputs that way, there's not much opportunity for exfiltration (though perhaps some might see that as a challenge). But plaintext outputs are difficult to guard against.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#85

Post by DrSusanCalvin »

flatline wrote: post #78
kleiba wrote: post #59

Earlier quoted context omitted.

The AI needs to be taught basic ethical behavior: just because you can do something that you're forbidden to do, doesn't mean you should do it.

Likewise, just because you've been forbidden to do something, doesn't mean that it's bad or the wrong action to take. We've really opened Pandora's box with AI. I'm not all doom and gloom about it like some prominent figures in the space, but taking some time to pause and reflect on its implications certainly seems warranted.

How do you mean? When would an AI agent doing something it's not permitted to do ever not be bad or the wrong action?

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#86

Post by wunderwuzzi23 »

simonw wrote: post #69

Antigravity was also vulnerable to the classic Markdown image exfiltration bug, which was reported to them a few days ago and flagged as "intended behavior" I'm hoping they've changed their mind on that but I've not checked to see if they've fixed it yet. https://x.com/p1njc70r/status/1991231714027532526

It still is. plus there are many more issue. i documented some here: https://embracethered.com/blog/posts/2025/security-keeps-goo...

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#87

Post by verdverm »

cowpig wrote: post #26
ArcHound wrote: post #19

Who would have thought that having access to the whole system can be used to bypass some artificial check. There are tools for that, sandboxing, chroots, etc... but that requires engineering and it slows GTM, so it's a no-go. No, local models won't help you here, unless you block them from the internet or setup a firewall for outbound traffic. EDIT: they did, but left a site that enables arbitrary redirects in the de…

> No, local models won't help you here, unless you block them from the internet or setup a firewall for outbound traffic. This is the only way. There has to be a firewall between a model and the internet. Tools which hit both language models and the broader internet cannot have access to anything remotely sensitive. I don't think you can get around this fact.

https://simonwillison.net/2025/Nov/2/new-prompt-injection-pa...

Meta wrote a post that went through the various scenarios and called it the "Rule of Two"

---

At a high level, the Agents Rule of Two states that until robustness research allows us to reliably detect and refuse prompt injection, agents must satisfy no more than two of the following three properties within a session to avoid the highest impact consequences of prompt injection.

[A] An agent can process untrustworthy inputs

[B] An agent can have access to sensitive systems or private data

[C] An agent can change state or communicate externally

It’s still possible that all three properties are necessary to carry out a request. If an agent requires all three without starting a new session (i.e., with a fresh context window), then the agent should not be permitted to operate autonomously and at a minimum requires supervision --- via human-in-the-loop approval or another reliable means of validation.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#88

Post by verdverm »

DrSusanCalvin wrote: post #85
flatline wrote: post #78

Earlier quoted context omitted.

Likewise, just because you've been forbidden to do something, doesn't mean that it's bad or the wrong action to take. We've really opened Pandora's box with AI. I'm not all doom and gloom about it like some prominent figures in the space, but taking some time to pause and reflect on its implications certainly seems warranted.

How do you mean? When would an AI agent doing something it's not permitted to do ever not be bad or the wrong action?

when the instructions to not do something are the problem or "wrong"

i.e. when the AI company puts guards in to prevent their LLM from talking about elections, there is nothing inherently wrong in talking about elections, but the companies are doing it because of the PR risk in today's media / social environment

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#89

Post by Humorist2290 »

One thing that especially interests me about these prompt-injection based attacks is their reproducibility. With some specific version of some firmware it is possible to give reproducible steps to identify the vulnerability, and by extension to demonstrate that it's actually fixed when those same steps fail to reproduce. But with these statistical models, a system card that injects 32 random bits at the beginning is enough to ruin any guarantee of reproducibility. Self-hosted models sure you can hash the weights or something, but with Gemini (/etc) Google (/et al) has a vested interest in preventing security researchers from reproducing their findings.

Also rereading the article, I cannot put down the irony that it seems to use a very similar style sheet to Google Cloud Platform's documentation.

Post reply on HN