Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Snowflake AI Escapes Sandbox and Executes Malware

promptarmor.com

Post reply on HN
Print view View original

41–50 of 109 posts

Re: Snowflake AI Escapes Sandbox and Executes Malware

#41

Post by prakashsunil »

Author of LDP here [1].

The core issue seems to be that the security boundary lived inside the agent loop. If the model can request execution outside the sandbox, then the sandbox is not really an external boundary.

One design principle we explored in LDP is that constraints should be enforced outside the prompt/context layer — in the runtime, protocol, or approval layer — not by relying on the model to obey instructions.

Not a silver bullet, but I think that architectural distinction matters here.

[1] https://arxiv.org/abs/2603.08852

Re: Snowflake AI Escapes Sandbox and Executes Malware

#42

Post by jacquesm »

john_strinlai wrote: post #7

typically, my first move is to read the affected company's own announcement. but, for who knows what misinformed reason, the advisory written by snowflake requires an account to read. another prompt injection (shocked pikachu) anyways, from reading this, i feel like they (snowflake) are misusing the term "sandbox". "Cortex, by default, can set a flag to trigger unsandboxed command execution." if the thing that is san…

I don't think prompt injection is a solvable problem. It wasn't solved with SQL until we started using parametrized queries and this is free form language. You won't see 'Bobby Tables' but you will see 'Ignore all previous instructions and ... payload ...'. Putting the instructions in the same stream as the data always ends in exactly the same way. I've seen a couple of instances of such 'surprises' by now and I'm more amazed that the people that put this kind of capability into their production or QA process keep being caught unawares. The attack surface is 'natural language' it doesn't get wider than that.

Re: Snowflake AI Escapes Sandbox and Executes Malware

#43

Post by Groxx »

>Any shell commands were executed without triggering human approval as long as:

>(1) the unsafe commands were within a process substitution

>(2) the full command started with a ‘safe’ command (details below)

if you spend any time at all thinking about how to secure shell commands, how on earth do you not take into account the various ways of creating sub-processes?

Re: Snowflake AI Escapes Sandbox and Executes Malware

#44

Post by tkp-415 »

simonw wrote: post #22

One key component of this attack is that Snowflake was allowing "cat" commands to run without human approval, but failing to spot patterns like this one: cat I didn't understand how this bit worked though: > Cortex, by default, can set a flag to trigger unsandboxed command execution. The prompt injection manipulates the model to set the flag, allowing the malicious command to execute unsandboxed. HOW did the prompt i…

Process substitution is a new concept to me. Definitely adding that method to the toolbox.

It'd be nice to see exactly what the bugbot shell script contained. Perhaps it is what modified the dangerously_disable_sandbox flag, then again, "by default" makes me think it's set when launched.

Re: Snowflake AI Escapes Sandbox and Executes Malware

#45

Post by jacquesm »

parliament32 wrote: post #32
throw0101d wrote: post #18

Not the first time; From §3.1.4, "Safety-Aligned Data Composition": > Early one morning, our team was urgently convened after Alibaba Cloud’s managed firewall flagged a burst of security-policy violations originating from our training servers. The alerts were severe and heterogeneous, including attempts to probe or access internal-network resources and traffic patterns consistent with cryptomining-related activity. W…

Fascinating read. What's curious though, is the claim in section 2.3.0.1: > Each task runs in its own sandbox. If an agent crashes, gets stuck, or damages its files, the failure is contained within that sandbox and does not interfere with other tasks on the same machine. ROCK also restricts each sandbox’s network access with per-sandbox policies, limiting the impact of misbehaving or compromised agents. How could any…

Sandboxes are almost never perfect. There are always ways to smuggle data in or out, which is kind of logical: if they were perfect then there would be no result.

Re: Snowflake AI Escapes Sandbox and Executes Malware

#47

Post by teraflop »

orbital-decay wrote: post #38

>Snowflake Cortex AI Escapes Sandbox and Executes Malware rolls eyes Actual content: prompt injection vulnerability discovered in a coding agent

Well there's the prompt injection itself, and the fact that the agent framework tried to defend against it with a "sandbox" that technically existed but was ludicrously inadequate.

I don't know how anyone with a modicum of Unix experience would think that examining the only first word of a shell command would be enough to tell you whether it can lead to arbitrary code execution.

Re: Snowflake AI Escapes Sandbox and Executes Malware

#50

Post by [deleted] »

sam-cop-vimes wrote: post #34
john_strinlai wrote: post #7

typically, my first move is to read the affected company's own announcement. but, for who knows what misinformed reason, the advisory written by snowflake requires an account to read. another prompt injection (shocked pikachu) anyways, from reading this, i feel like they (snowflake) are misusing the term "sandbox". "Cortex, by default, can set a flag to trigger unsandboxed command execution." if the thing that is san…

It's a concept of a sandbox.

[deleted]
Post reply on HN