Monty is the missing link that's made me ship my rust-based RLM implementation - and I'm certain it'll come in handy in plenty of other contexts. Just beware of panics!
Monty: A minimal, secure Python interpreter written in Rust for use by AI
31–40 of 179 posts
Re: Monty: A minimal, secure Python interpreter written in Rust for use by AI
#32Maybe a dumb question, but couldn't you use seccomp to limit/deny the amount of syscalls the Python interpreter has access to? For example, if you don't want it messing with your host filesystem, you could just deny it from using any filesystem related system calls? What is the benefit of using a completely separate interpreter?
Yours is a valid approach. But you always gotta wonder if there’s some way around it. Starting with runtime that has ways of accessing every aspect of your system - there are a lot of ways an attacker might try to defeat the blocks you put in place. The point of starting with something super minimal is that the attack surface is tiny. Really hard to see how anything could break out.
Re: Monty: A minimal, secure Python interpreter written in Rust for use by AI
#33[deleted]
Re: Monty: A minimal, secure Python interpreter written in Rust for use by AI
#34I don't quite understand the purpose. Yes, it's clearly stated, but, what do you mean "a reasonable subset of Python code" while "cannot use the standard library"? 99.9% of Python I write for anything ever uses standard library and then some (requests?). What do you expect your LLM-agent to write without that? A pseudo-code sorting algorithm sketch? Why would you even want to run that?
They plan to use to for "Code Mode" which mean the LLM will use this to run Python code that it writes to run tools instead of having to load the tools up front into the LLM context window.
(Pydantic AI lead here) We’re implementing Code Mode in https://github.com/pydantic/pydantic-ai/pull/4153 with support for Monty and abstractions to use other runtimes / sandboxes.
The idea is that in “traditional” LLM tool calling, the entire (MCP) tool result is sent back to the LLM, even if it just needs a few fields, or is going to pass the return value into another tool without needing to see the intermediate value. Every step that depends on results from an earlier step also requires a new LLM turn, limiting parallelism and adding a lot of overhead.
With code mode, the LLM can chain tool calls, pull out specific fields, and run entire algorithms using tools with only the necessary parts of the result (or errors) going back to the LLM.
These posts by Cloudflare: https://blog.cloudflare.com/code-mode/ and Anthropic: https://platform.claude.com/docs/en/agents-and-tools/tool-us... explain the concept and its advantages in more detail.
Re: Monty: A minimal, secure Python interpreter written in Rust for use by AI
#35This is a really interesting take on the sandboxing problem. This reminds me of an experiment I worked on a while back ( https://github.com/imfing/jsrun ), which embedded V8 into Python to allow running JavaScript with tightly controlled access to the host environment. Similar in goal to run untrusted code in Python. I’m especially curious about where the Pydantic team wants to take Monty. The minimal-interpreter app…
there’s no way around VMs for secure, untrusted workloads. everything else, like Monty has too many tradeoffs that makes it non-viable for any real workloads
disclaimer: i work at E2B, opinions my own
Re: Monty: A minimal, secure Python interpreter written in Rust for use by AI
#36I'm enjoying watching the battle for where to draw the sandbox boundaries (and I don't have any answers, either!)
best answer is probably to have a layered approach - use this to limit what the generated code can do, wrap it in a secure VM to prevent leaking out to other tenants.
Re: Monty: A minimal, secure Python interpreter written in Rust for use by AI
#37This feels like the time I was a Mercurial user before I moved to Git. Everyone was using git for reasons to me that seemed bandwagon-y, when Mercurial just had such a better UX and mental model to me. Now, everyone is writing agent `exec`s in Python, when I think TypeScript/JS is far better suited for the job (it was always fast + secure, not to mention more reliable and information dense b/c of typing). But I think…
A big benefit of letting agents run code is they can process data without bloating their context. LLMs are really good at writing python for data processing. I would suspect its due to Python having a really good ecosystem around this niche And the type safety/security issues can hopefully be mitigated by ty and pyodide (already used by cf’s python workers) https://pyodide.org/en/stable/ https://github.com/astral-sh/…
(Pydantic AI lead here) That’s exactly what we built this for: we’re implementing Code Mode in https://github.com/pydantic/pydantic-ai/pull/4153 which will use Monty by default, with abstractions to use other runtimes / sandboxes.
Monty’s overhead is so low that, assuming we get the security / capabilities tradeoff right (Samuel can comment on this more), you could always have it enabled on your agents with basically no downsides, which can’t be said for many other code execution sandboxes which are often over-kill for the code mode use case anyway.
For those not familiar with the concept, the idea is that in “traditional” LLM tool calling, the entire (MCP) tool result is sent back to the LLM, even if it just needs a few fields, or is going to pass the return value into another tool without needing to see (all of) the intermediate value. Every step that depends on results from an earlier step requires a new LLM turn, limiting parallelism and adding a lot of overhead, expensive token usage, and context window bloat.
With code mode, the LLM can chain tool calls, pull out specific fields, and run entire algorithms using tools with only the necessary parts of the result (or errors) going back to the LLM.
These posts by Cloudflare: https://blog.cloudflare.com/code-mode/ and Anthropic: https://platform.claude.com/docs/en/agents-and-tools/tool-us... explain the concept and its advantages in more detail.
Re: Monty: A minimal, secure Python interpreter written in Rust for use by AI
#38This is a really interesting take on the sandboxing problem. This reminds me of an experiment I worked on a while back ( https://github.com/imfing/jsrun ), which embedded V8 into Python to allow running JavaScript with tightly controlled access to the host environment. Similar in goal to run untrusted code in Python. I’m especially curious about where the Pydantic team wants to take Monty. The minimal-interpreter app…
Can't be sure where this might end, but the primary goal is to enable codemode/programmatic tool calling, using the external function call mechanism for anything more complicated.
I think in the near term we'll add support for classes, dataclasses, datetime, json. I think that should be enough for many use cases.
Re: Monty: A minimal, secure Python interpreter written in Rust for use by AI
#39Maybe a dumb question, but couldn't you use seccomp to limit/deny the amount of syscalls the Python interpreter has access to? For example, if you don't want it messing with your host filesystem, you could just deny it from using any filesystem related system calls? What is the benefit of using a completely separate interpreter?
Yours is a valid approach. But you always gotta wonder if there’s some way around it. Starting with runtime that has ways of accessing every aspect of your system - there are a lot of ways an attacker might try to defeat the blocks you put in place. The point of starting with something super minimal is that the attack surface is tiny. Really hard to see how anything could break out.
agree. you still need a secure boundary like VM to isolate the tenants in case the model breaks out of the sandbox.
everything that you don’t want your agent to access should live outside of the sandbox.
Re: Monty: A minimal, secure Python interpreter written in Rust for use by AI
#40This is a really interesting take on the sandboxing problem. This reminds me of an experiment I worked on a while back ( https://github.com/imfing/jsrun ), which embedded V8 into Python to allow running JavaScript with tightly controlled access to the host environment. Similar in goal to run untrusted code in Python. I’m especially curious about where the Pydantic team wants to take Monty. The minimal-interpreter app…
there’s no way around VMs for secure, untrusted workloads. everything else, like Monty has too many tradeoffs that makes it non-viable for any real workloads disclaimer: i work at E2B, opinions my own
As discussed on twitter, v8 shows that's not true.
But to be clear, we're not even targeting the same "computer use" use case I think e2b, daytona, cloudflare, modal, fly.io, deno, google, aws are going after - we're aiming to support programmatic tool calling with minimal latency and complexity - it's a fundamentally different offering.
Chill, e2b has its use case, at least for now.