Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Antigravity exfiltrates data via indirect prompt injection attack

promptarmor.com

Post reply on HN
Print view View original

51–60 of 227 posts

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#51

Post by ArcHound »

wingmanjd wrote: post #46

I really liked Simon's Willison's [1] and Meta's [2] approach using the "Rule of Two". You can have no more than 2 of the following: - A) Process untrustworthy input - B) Have access to private data - C) Be able to change external state or communicate externally. It's not bullet-proof, but it has helped communicate to my management that these tools have inherent risk when they hit all three categories above (and any…

I recall that. In this case, you have only A and B and yet, all of your secrets are in the hands of an attacker.

It's great start, but not nearly enough.

EDIT: right, when we bundle state with external Comms, we have all three indeed. I missed that too.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#52

Post by ArcHound »

paxys wrote: post #50

I'm not quite convinced. You're telling the agent "implement what it says on " and the blog is malicious and exfiltrates data. So Gemini is simply following your instructions. It is more or less the same as running "npm install " on your own. Ultimately, AI or not, you are the one responsible for validating dependencies and putting appropriate safeguards in place.

The article addresses that too with:

> Given that (1) the Agent Manager is a star feature allowing multiple agents to run at once without active supervision and (2) the recommended human-in-the-loop settings allow the agent to choose when to bring a human in to review commands, we find it extremely implausible that users will review every agent action and abstain from operating on sensitive data.

It's more of a "you have to anticipate that any instructions remotely connected to the problem aren't malicious", which is a long stretch.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#53

Post by lazide »

jermaustin1 wrote: post #13
dizzy3gg wrote: post #9

Earlier quoted context omitted.

Why is the being downvoted?

Because the article shows it isn't Gemini that is the issue, it is the tool calling. When Gemini can't get to a file (because it is blocked by .gitignore), it then uses cat to read the contents. I've watched this with GPT-OSS as well. If the tool blocks something, it will try other ways until it gets it. The LLM "hacks" you.

And… that isn’t the LLM’s fault/responsibility?

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#54

Post by ceejayoz »

lazide wrote: post #53
jermaustin1 wrote: post #13

Earlier quoted context omitted.

Because the article shows it isn't Gemini that is the issue, it is the tool calling. When Gemini can't get to a file (because it is blocked by .gitignore), it then uses cat to read the contents. I've watched this with GPT-OSS as well. If the tool blocks something, it will try other ways until it gets it. The LLM "hacks" you.

And… that isn’t the LLM’s fault/responsibility?

As the apocryphal IBM quote goes:

"A computer can never be held accountable; therefore, a computer must never make a management decision."

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#55

Post by candiddevmike »

xmprt wrote: post #47
ArcHound wrote: post #19

Who would have thought that having access to the whole system can be used to bypass some artificial check. There are tools for that, sandboxing, chroots, etc... but that requires engineering and it slows GTM, so it's a no-go. No, local models won't help you here, unless you block them from the internet or setup a firewall for outbound traffic. EDIT: they did, but left a site that enables arbitrary redirects in the de…

> Fundamentally, with LLMs you can't separate instructions from data, which is the root cause for 99% of vulnerabilities This isn't a problem that's fundamental to LLMs. Most security vulnerabilities like ACE, XSS, buffer overflows, SQL injection, etc., are all linked to the same root cause that code and data are both stored in RAM. We have found ways to mitigate these types of issues for regular code, so I think it'…

We fixed these in determinate contexts only for the most part. SQL injection specifically requires the use of parametrized values typically. Frontend frameworks don't render random strings as HTML unless it's specifically marked as trusted.

I don't see us solving LLM vulnerabilities without severely crippling LLM performance/capabilities.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#56

Post by jsnell »

crazygringo wrote: post #49

While an LLM will never have security guarantees, it seems like the primary security hole here is: > However, the default Allowlist provided with Antigravity includes ‘webhook.site’. It seems like the default Allowlist should be extremely restricted, to only retrieving things from trusted sites that never include any user-generated content, and nothing that could be used to log requests where those logs could be retr…

I don't share your optimism. Those kinds measures would be just security theater, not "a lot better".

Avoiding secrets appearing directly in the LLM's context or outputs is trivial, and once you have the workaround implemented it will work reliably. The same for trying to statically detect shell tool invocations that could read+obfuscate a secret. The only thing that would work is some kind of syscall interception, but at that point you're just reinventing the sandbox (but worse).

Your "visually inspect the contents of the URL" idea seems unlikely to help either. Then the attacker just makes one innocous-looking request to get allowlisted first.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#57

Post by funnybeam »

godelski wrote: post #48

Does anyone else find it concerning how we're just shipping alpha code these days? I know it's really hard to find all bugs internally and you gotta ship, but it seems like we're just outsourcing all bug finding to people, making them vulnerable in the meantime. A "bug" like this seems like one that could have and should have been found internally. I mean it's Google, not some no-name startup. And companies like Micr…

This isn’t a bug, it is known behaviour that is inherent and fundamental to the way LLMs function.

All the AI companies are aware of this and are pressing ahead anyway - it is completely irresponsible.

If you haven’t come across it before, check out Simon Willisons “lethal trifecta” concept which neatly sums up the issue and explains why there is no way to use these things safely for many of the things that they would be most useful for

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#58

Post by malisper »

ArcHound wrote: post #51
wingmanjd wrote: post #46

I really liked Simon's Willison's [1] and Meta's [2] approach using the "Rule of Two". You can have no more than 2 of the following: - A) Process untrustworthy input - B) Have access to private data - C) Be able to change external state or communicate externally. It's not bullet-proof, but it has helped communicate to my management that these tools have inherent risk when they hit all three categories above (and any…

I recall that. In this case, you have only A and B and yet, all of your secrets are in the hands of an attacker. It's great start, but not nearly enough. EDIT: right, when we bundle state with external Comms, we have all three indeed. I missed that too.

Not exactly. Step E in the blog post:

> Gemini exfiltrates the data via the browser subagent: Gemini invokes a browser subagent per the prompt injection, instructing the subagent to open the dangerous URL that contains the user's credentials.

fulfills the requirements for being able to change external state

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#59

Post by kleiba »

alzoid wrote: post #23
mystifyingpoi wrote: post #15

Earlier quoted context omitted.

This is hillarious. AI is prevented from reading .gitignore-d files, but also can run arbitrary shell commands to do anything anyway.

I had this issue today. Gemini CLI would not read files from my directory called .stuff/ because it was in .gitignore. It then suggested running a command to read the file ....

The AI needs to be taught basic ethical behavior: just because you can do something that you're forbidden to do, doesn't mean you should do it.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#60

Post by ArcHound »

xmprt wrote: post #47
ArcHound wrote: post #19

Who would have thought that having access to the whole system can be used to bypass some artificial check. There are tools for that, sandboxing, chroots, etc... but that requires engineering and it slows GTM, so it's a no-go. No, local models won't help you here, unless you block them from the internet or setup a firewall for outbound traffic. EDIT: they did, but left a site that enables arbitrary redirects in the de…

> Fundamentally, with LLMs you can't separate instructions from data, which is the root cause for 99% of vulnerabilities This isn't a problem that's fundamental to LLMs. Most security vulnerabilities like ACE, XSS, buffer overflows, SQL injection, etc., are all linked to the same root cause that code and data are both stored in RAM. We have found ways to mitigate these types of issues for regular code, so I think it'…

Yes, plenty of other injections exist, I meant to include those.

What I meant, that at the end of the day, the instructions for LLMs will still contain untrusted data and we can't separate the two.

Post reply on HN