Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Antigravity exfiltrates data via indirect prompt injection attack

promptarmor.com

Post reply on HN
Print view View original

11–20 of 227 posts

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#11

Post by ArcHound »

serial_dev wrote: post #5

> Gemini is not supposed to have access to .env files in this scenario (with the default setting ‘Allow Gitignore Access > Off’). However, we show that Gemini bypasses its own setting to get access and subsequently exfiltrate that data. They pinky promised they won’t use something, and the only reason we learned about it is because they leaked the stuff they shouldn’t even be able to see?

When I read this I thought about a Dev frustrated with a restricted environment saying "Well, akschually.."

So more of a Gemini initiated bypass of it's own instructions than malicious Google setup.

Gemini can't see it, but it can instruct cat to output it and read the output.

Hilarious.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#12

Post by api »

mkagenius wrote: post #4

Sooner or later I believe, there will be models which can be deployed locally on your mac and are as good as say Sonnet 4.5. People should shift to completely local at that point. And use sandbox for executing code generated by llm. Edit: "completely local" meant not doing any network calls unless specifically approved. When llm calls are completely local you just need to monitor a few explicit network calls to be su…

Can't find 4.5, but 3.5 Sonnet is apparently about 175 billion parameters. At 8-bit quantization that would fit on a box with 192 gigs of unified RAM.

The most RAM you can currently get in a MacBook is 128 gigs, I think, and that's a pricey machine, but it could run such a model at 4-bit or 5-bit quantization.

As time goes on it only gets cheaper, so yes this is possible.

The question is whether bigger and bigger models will keep getting better. What I'm seeing suggests we will see a plateau, so probably not forever. Eventually affordable endpoint hardware will catch up.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#13

Post by jermaustin1 »

dizzy3gg wrote: post #9
mkagenius wrote: post #4

Sooner or later I believe, there will be models which can be deployed locally on your mac and are as good as say Sonnet 4.5. People should shift to completely local at that point. And use sandbox for executing code generated by llm. Edit: "completely local" meant not doing any network calls unless specifically approved. When llm calls are completely local you just need to monitor a few explicit network calls to be su…

Why is the being downvoted?

Because the article shows it isn't Gemini that is the issue, it is the tool calling. When Gemini can't get to a file (because it is blocked by .gitignore), it then uses cat to read the contents.

I've watched this with GPT-OSS as well. If the tool blocks something, it will try other ways until it gets it.

The LLM "hacks" you.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#14

Post by bo1024 »

serial_dev wrote: post #5

> Gemini is not supposed to have access to .env files in this scenario (with the default setting ‘Allow Gitignore Access > Off’). However, we show that Gemini bypasses its own setting to get access and subsequently exfiltrate that data. They pinky promised they won’t use something, and the only reason we learned about it is because they leaked the stuff they shouldn’t even be able to see?

As you see later, it uses cat to dump the contents of a file it’s not allowed to open itself.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#15

Post by mystifyingpoi »

serial_dev wrote: post #5

> Gemini is not supposed to have access to .env files in this scenario (with the default setting ‘Allow Gitignore Access > Off’). However, we show that Gemini bypasses its own setting to get access and subsequently exfiltrate that data. They pinky promised they won’t use something, and the only reason we learned about it is because they leaked the stuff they shouldn’t even be able to see?

This is hillarious. AI is prevented from reading .gitignore-d files, but also can run arbitrary shell commands to do anything anyway.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#16

Post by NitpickLawyer »

dizzy3gg wrote: post #9
mkagenius wrote: post #4

Sooner or later I believe, there will be models which can be deployed locally on your mac and are as good as say Sonnet 4.5. People should shift to completely local at that point. And use sandbox for executing code generated by llm. Edit: "completely local" meant not doing any network calls unless specifically approved. When llm calls are completely local you just need to monitor a few explicit network calls to be su…

Why is the being downvoted?

Because it misses the point. The problem is not the model being in a cloud. The problem is that as soon as "untrusted inputs" (i.e. web content) touch your LLM context, you are vulnerable to data exfil. Running the model locally has nothing to do with avoiding this. Nor does "running code in a sandbox", as long as that sandbox can hit http / dns / whatever.

The main problem is that LLMs share both "control" and "data" channels, and you can't (so far) disambiguate between the two. There are mitigations, but nothing is 100% safe.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#17

Post by yodon »

KK7NIL wrote: post #8
mkagenius wrote: post #4

Sooner or later I believe, there will be models which can be deployed locally on your mac and are as good as say Sonnet 4.5. People should shift to completely local at that point. And use sandbox for executing code generated by llm. Edit: "completely local" meant not doing any network calls unless specifically approved. When llm calls are completely local you just need to monitor a few explicit network calls to be su…

If you read the article you'd notice that running an LLM locally would not fix this vulnerability.

From the HN guidelines[0]:

>Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that".

[0]: https://news.ycombinator.com/newsguidelines.html

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#19

Post by ArcHound »

Who would have thought that having access to the whole system can be used to bypass some artificial check.

There are tools for that, sandboxing, chroots, etc... but that requires engineering and it slows GTM, so it's a no-go.

No, local models won't help you here, unless you block them from the internet or setup a firewall for outbound traffic. EDIT: they did, but left a site that enables arbitrary redirects in the default config.

Fundamentally, with LLMs you can't separate instructions from data, which is the root cause for 99% of vulnerabilities.

Security is hard man, excellent article, thoroughly enjoyed.

Re: Google Antigravity exfiltrates data via indirect prompt injection attack

#20

Post by pennomi »

KK7NIL wrote: post #8
mkagenius wrote: post #4

Sooner or later I believe, there will be models which can be deployed locally on your mac and are as good as say Sonnet 4.5. People should shift to completely local at that point. And use sandbox for executing code generated by llm. Edit: "completely local" meant not doing any network calls unless specifically approved. When llm calls are completely local you just need to monitor a few explicit network calls to be su…

If you read the article you'd notice that running an LLM locally would not fix this vulnerability.

Right, you’d have to deny the LLM access to online resources AND all web-capable tools… which severely limits an agent’s capabilities.
Post reply on HN