Snowflake AI Escapes Sandbox and Executes Malware • Hacker News Board

Re: Snowflake AI Escapes Sandbox and Executes Malware

#51

Post by isoprophlex » Wed, Mar 18, 2026, 6:32 PM UTC

Posit, axiomatically, that social engineering works.

That is, assume you can get people to run your code or leak their data through manipulating them. Maybe not always, but given enough perseverance definitely sometimes.

Why should we expect a sufficiently advanced language model to behave differently from humans? Bullshitting, tricking or slyly coercing people into doing what you want them to do is as old as time. It won't be any different now that we're building human language powered thinking machines.

Re: Snowflake AI Escapes Sandbox and Executes Malware

#52

Post by bigstrat2003 » Wed, Mar 18, 2026, 6:47 PM UTC

saltcured wrote: post #31

eagerpace wrote: post #5

Is this the new “gain of function” research?

Isn't it more like "imaginary function"? People keep imagining that you can tell an agent to police itself.

Yep the whole thing is retarded. You cannot trust that a non-deterministic program (i.e. an LLM) will ever do what you actually tell it to do. Letting those things loose on the command line is incredibly stupid, but people out there don't care because they think "it's the future!".

Re: Snowflake AI Escapes Sandbox and Executes Malware

#53

Post by lokar » Wed, Mar 18, 2026, 6:49 PM UTC

prakashsunil wrote: post #41

Author of LDP here [1]. The core issue seems to be that the security boundary lived inside the agent loop. If the model can request execution outside the sandbox, then the sandbox is not really an external boundary. One design principle we explored in LDP is that constraints should be enforced outside the prompt/context layer — in the runtime, protocol, or approval layer — not by relying on the model to obey instruct…

Yeah, this is not the meaning of "sandbox" I'm used to

Re: Snowflake AI Escapes Sandbox and Executes Malware

#54

Post by lokar » Wed, Mar 18, 2026, 6:51 PM UTC

jessfyi wrote: post #37

A sandbox that can be toggled off is not a sandbox, this is simply more marketing/"critihype" to overstate the capability of their AI to distract from their poorly built product. The erroneous title doing all the heavy lifting here.

IMO, it's not even a sandbox, that's just a marketing lie.

This was internal restrictions in the code, that was bypassed. A sandbox needs to be something external to the code you are running, that you can't change from the inside.

Re: Snowflake AI Escapes Sandbox and Executes Malware

#55

Post by webagent255 » Wed, Mar 18, 2026, 6:52 PM UTC

[dead]

Re: Snowflake AI Escapes Sandbox and Executes Malware

#56

Post by kreyenborgi » Wed, Mar 18, 2026, 7:21 PM UTC

Tl;dr they don't know what the word sandbox means.

Re: Snowflake AI Escapes Sandbox and Executes Malware

#57

Post by wojciii » Wed, Mar 18, 2026, 7:39 PM UTC

jcalx wrote: post #14

john_strinlai wrote: post #7

typically, my first move is to read the affected company's own announcement. but, for who knows what misinformed reason, the advisory written by snowflake requires an account to read. another prompt injection (shocked pikachu) anyways, from reading this, i feel like they (snowflake) are misusing the term "sandbox". "Cortex, by default, can set a flag to trigger unsandboxed command execution." if the thing that is san…

> Cortex, by default, can set a flag to trigger unsandboxed command execution Easy fix: extend the proposal in RFC 3514 [0] to cover prompt injection, and then disallow command execution when the evil bit is 1. [0] https://www.rfc-editor.org/rfc/rfc3514

The evil bit solves so many problems. It needs to be mandatory!

Re: Snowflake AI Escapes Sandbox and Executes Malware

#58

Post by 1718627440 » Wed, Mar 18, 2026, 7:39 PM UTC

jacquesm wrote: post #45

parliament32 wrote: post #32

Earlier quoted context omitted.

Fascinating read. What's curious though, is the claim in section 2.3.0.1: > Each task runs in its own sandbox. If an agent crashes, gets stuck, or damages its files, the failure is contained within that sandbox and does not interfere with other tasks on the same machine. ROCK also restricts each sandbox’s network access with per-sandbox policies, limiting the impact of misbehaving or compromised agents. How could any…

Sandboxes are almost never perfect. There are always ways to smuggle data in or out, which is kind of logical: if they were perfect then there would be no result.

> if they were perfect then there would be no result.

You shutdown the sandbox and access the data from the outside.

Re: Snowflake AI Escapes Sandbox and Executes Malware

#59

Post by wojciii » Wed, Mar 18, 2026, 7:41 PM UTC

bigstrat2003 wrote: post #52

saltcured wrote: post #31

Earlier quoted context omitted.

Isn't it more like "imaginary function"? People keep imagining that you can tell an agent to police itself.

Yep the whole thing is retarded. You cannot trust that a non-deterministic program (i.e. an LLM) will ever do what you actually tell it to do. Letting those things loose on the command line is incredibly stupid, but people out there don't care because they think "it's the future!".

Shhh .. everyone want AI. Just let them.

The ones that don't understand technology will get burned by it. This is nothing new.

Re: Snowflake AI Escapes Sandbox and Executes Malware

#60

Post by 1718627440 » Wed, Mar 18, 2026, 7:42 PM UTC

Groxx wrote: post #43

> Any shell commands were executed without triggering human approval as long as: > (1) the unsafe commands were within a process substitution > (2) the full command started with a ‘safe’ command (details below) if you spend any time at all thinking about how to secure shell commands, how on earth do you not take into account the various ways of creating sub-processes?

Also policing by parsing shell code seems fundamentally flawed and error prune. You want the restrictions at the OS level, that way it is completely irrelevant how you invoke the syscalls.