Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monty: A minimal, secure Python interpreter written in Rust for use by AI

github.com

Post reply on HN
Print view View original

151–160 of 179 posts

Re: Monty: A minimal, secure Python interpreter written in Rust for use by AI

#151

Post by zahlman »

63stack wrote: post #130
miki123211 wrote: post #94

Earlier quoted context omitted.

3 reasons why Python is much better than JS for this IMO. 1. Large built-in standard library (CSV, sqlite3, xml/json, zipfile). 2. In Python, whatever the LLM is likely to do will probably work. In JS, you have the Node / Deno split, far too many libraries that do the same thing (XMLHTTPRequest / Axios / fetch), many mutually-incompatible import syntaxes (E.G. compare tsx versus Node's native ts execution), and featu…

>In Python, whatever the LLM is likely to do will probably work. Do you not realize how this sounds? >many mutually-incompatible import syntaxes Do you think there are 22 competing package managers in python because the package/import system "just works"?

> Do you think there are 22 competing package managers in python because the package/import system "just works"?

There aren't; a large fraction of tools people mention in this context aren't actually package managers and don't try to be package managers. Sometimes people even conflate standards and config files with tools. It's really amazing how much FUD there is around it.

But more importantly, there is no such thing as "the package/import system". Packaging is one thing, and the language's import system is a completely different thing.

And none of that actually bears on the LLM's ability to choose libraries and figure out language syntax and APIs. For that matter, you don't have to let it set up the environment (or change your existing setup) if you don't want to.

Re: Monty: A minimal, secure Python interpreter written in Rust for use by AI

#153

Post by zahlman »

andai wrote: post #136

Doesn't the agent already have bash though? My current security model is to give it a separate Linux user. So it can blow itself up and... I think that's about it?

> Doesn't the agent already have bash though?

You don't have to give it bash, depending on your tools at least.

> So it can blow itself up and... I think that's about it?

And exfiltrate data via the Internet, fill up disk space...

Re: Monty: A minimal, secure Python interpreter written in Rust for use by AI

#154

Post by oofbey »

DouweM wrote: post #37
shoeb00m wrote: post #16

Earlier quoted context omitted.

A big benefit of letting agents run code is they can process data without bloating their context. LLMs are really good at writing python for data processing. I would suspect its due to Python having a really good ecosystem around this niche And the type safety/security issues can hopefully be mitigated by ty and pyodide (already used by cf’s python workers) https://pyodide.org/en/stable/ https://github.com/astral-sh/…

(Pydantic AI lead here) That’s exactly what we built this for: we’re implementing Code Mode in https://github.com/pydantic/pydantic-ai/pull/4153 which will use Monty by default, with abstractions to use other runtimes / sandboxes. Monty’s overhead is so low that, assuming we get the security / capabilities tradeoff right (Samuel can comment on this more), you could always have it enabled on your agents with basically…

Just want to say Kudos to you and the team. This is a brilliantly conceived chunk of functionality that IMHO hits exactly a sweet spot I didn’t realize was missing. I’m working on a chat bot system now and definitely plan to incorporate Monty into it for all the reasons y’all foresaw.

Thank you!!

Re: Monty: A minimal, secure Python interpreter written in Rust for use by AI

#155

Post by spacedatum »

wiseowise wrote: post #96
spacedatum wrote: post #91

Earlier quoted context omitted.

It is a tradeoff, but I prefer my checks at compile time to runtime. Python can be brittle and silently wrong.

What kind of type checking do you think Rust does at runtime?

Google it and try it yourself.

Re: Monty: A minimal, secure Python interpreter written in Rust for use by AI

#156

Post by cyanydeez »

qwertox wrote: post #133
jstanley wrote: post #123

Earlier quoted context omitted.

> But it doesn't matter, because LLMs that try to use a class will get an error message and rewrite their code to not use classes instead. This is true in a sense, but every little papercut at the lower levels of abstraction degrades performance at higher levels as the LLM needs to spend its efforts on hacking around jank in the Python interpreter instead of solving the real problem.

It is a workaround, so we can assume that this will be temporary and in the future the ai will then start using them once it can. Probably just like we would do.

Thw entire AI stack is built on a lot of "assumes" about intelligent selection.

Reminds of evolutionary debate. Whats important is just because something can learn to adapt doesnt mean theyll find an optimized adaption, nor will they continually refine it.

As far as i can tell AI will only solve problems well where the problem space is properly defined. Most people wont know how to do that.

Re: Monty: A minimal, secure Python interpreter written in Rust for use by AI

#157

Post by the_harpia_io »

zahlman wrote: post #152
the_harpia_io wrote: post #135

[flagged]

My understanding is that "the class restriction" isn't trying to implement any kind of security boundary — they just haven't managed to implement support yet.

[flagged]

Re: Monty: A minimal, secure Python interpreter written in Rust for use by AI

#159

Post by bonoboTP »

falcor84 wrote: post #150
saberience wrote: post #120

Earlier quoted context omitted.

I really don't understand the use-case here. My models are writing code all day in 3/4 different languages, why would I want to: a) Restrict them to Python b) Restrict them to a cutdown, less-useful version of Python? My models write me Typescript and C# and Python all day with zero issues. Why do I need this?

For extremely rapid iteration - they can run a quick script with this in under 1ms - it removes a significant bottleneck, especially for math-heavy reasoning

Not sure if I get it, but it seems to me that this is not for "producing code" eg for your projects or doing things on your computer but essentially for supplementing its own thinking process. It runs this python code to count how many letters R in strawberry if you ask that, or does quick math, quick sorting and simple well defined tasks like this that are needed for answering the query or doing the job you asked to do. It's not indended to be read by the user and it's not a "deliverable" for the user.

Re: Monty: A minimal, secure Python interpreter written in Rust for use by AI

#160

Post by falcor84 »

bonoboTP wrote: post #159
falcor84 wrote: post #150

Earlier quoted context omitted.

For extremely rapid iteration - they can run a quick script with this in under 1ms - it removes a significant bottleneck, especially for math-heavy reasoning

Not sure if I get it, but it seems to me that this is not for "producing code" eg for your projects or doing things on your computer but essentially for supplementing its own thinking process. It runs this python code to count how many letters R in strawberry if you ask that, or does quick math, quick sorting and simple well defined tasks like this that are needed for answering the query or doing the job you asked to…

I'm working on a system where user requests queue up agentic runs that do a bit of analytical reasoning and then return a response to the user, and this interpreter can possibly help me significantly reduce the runtime of these jobs.
Post reply on HN