Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Scan your AI agents for dangerous capabilities

github.com

Post reply on HN
Print view View original

11–20 of 25 posts

Re: Show HN: Scan your AI agents for dangerous capabilities

#11

Post by __MatrixMan__ »

smashini wrote: post #8
smashini wrote: post #6

Earlier quoted context omitted.

I’d say the biggest difference would be: 1. Parameter-aware rules: OS permissions don’t know your application logic. (How would you tell OS permissions not to let your AI to trade on over 1M dollars) 2. You can’t easily model multi-pary and RBAC. 3. Agents call remote APIs for alot of those tools. Native OS doesn’t really parse web traffic to decide if a request is safe or not. OS sandboxing is good for host security…

So Linux can prevent an agent from opening /etc/passwd. Linux cannot stop an agent from calling: POST /wire-transfer amount=5,000,000

You'd do that with a container and a layer 7 egress proxy rule e.g. mitmproxy.

Sure it's work to build such things, but building restraints into the app feels more reliable than playing whack-a-mole with scanner results.

Operating systems can probably do better to meet this need (e.g. capability based ones like fuchsia) but even as is their rules just feel so much more binding.

Re: Show HN: Scan your AI agents for dangerous capabilities

#13

Post by quixoticaxolotl »

__MatrixMan__ wrote: post #3

Why build separate frameworks for this kind of thing when your operating system is right there? You can make a file called "orders" and you can run your agent as a user with write access to that file, or as one that doesn't, and then you don't need scans or audits to tell you whether the agent can create orders or not, you can just take your operating system's word for it. Is there anything all this bolt-on AI securi…

One benefit is that this can run in serverless / sandboxed containers where OS primitives are not exposed or heavily limited. I immediately thought of Cloudflare Workers, which runs on V8 and exposes WASM-only interfaces, using Workers AI.

Further, servers still have hosting value, but any business running agents is almost certainly going to want a sandbox that limits what code runs for agentic work, so targeting _sandbox_ environments is probably the better bet long-term. And, yes, you could implement your proposal in any chroot jail or gvisor, but nobody wants to get their hands dirty finnicking with that - programmatic access control beats file-based access control for the simple reason it's managed for you.

If anything, my critique of OP's implementation is actually the opposite of yours: they've chosen the right primitive and layer, but people really need contextual access control rather than RBAC. Sort of like ongoing zero trust. If it was possible to inspect the context, decide if it was a bad idea to allow the tool call, without exposing the decider to untrusted context, you could have something that really changes things.

Re: Show HN: Scan your AI agents for dangerous capabilities

#14

Post by smashini »

__MatrixMan__ wrote: post #11
smashini wrote: post #8

Earlier quoted context omitted.

So Linux can prevent an agent from opening /etc/passwd. Linux cannot stop an agent from calling: POST /wire-transfer amount=5,000,000

You'd do that with a container and a layer 7 egress proxy rule e.g. mitmproxy. Sure it's work to build such things, but building restraints into the app feels more reliable than playing whack-a-mole with scanner results. Operating systems can probably do better to meet this need (e.g. capability based ones like fuchsia) but even as is their rules just feel so much more binding.

The scanner is just one part of the codebase, good for maintaining quality in a pipeline.

There is also @makerchecker/embedded, which has runtime permission primitives you wrap around the agent's actions, so the restraints live in the app exactly like you're describing

Less whack-a-mole, more wrap the risky calls and they're bound

Re: Show HN: Scan your AI agents for dangerous capabilities

#15

Post by seethishat »

smashini wrote: post #8
smashini wrote: post #6

Earlier quoted context omitted.

I’d say the biggest difference would be: 1. Parameter-aware rules: OS permissions don’t know your application logic. (How would you tell OS permissions not to let your AI to trade on over 1M dollars) 2. You can’t easily model multi-pary and RBAC. 3. Agents call remote APIs for alot of those tools. Native OS doesn’t really parse web traffic to decide if a request is safe or not. OS sandboxing is good for host security…

So Linux can prevent an agent from opening /etc/passwd. Linux cannot stop an agent from calling: POST /wire-transfer amount=5,000,000

Any LSM can stop that POST. SELinux, AppArmor, Tomoyo, etc. They are built-in to the kernel. You just need to know what you are doing to use them.

Re: Show HN: Scan your AI agents for dangerous capabilities

#16

Post by bureado »

__MatrixMan__ wrote: post #3

Why build separate frameworks for this kind of thing when your operating system is right there? You can make a file called "orders" and you can run your agent as a user with write access to that file, or as one that doesn't, and then you don't need scans or audits to tell you whether the agent can create orders or not, you can just take your operating system's word for it. Is there anything all this bolt-on AI securi…

The user separation isn’t even necessary, as far as I’ve seen in the projects in https://github.com/bureado/awesome-agent-runtime-security

Re: Show HN: Scan your AI agents for dangerous capabilities

#18

Post by smashini »

bureado wrote: post #16
__MatrixMan__ wrote: post #3

Why build separate frameworks for this kind of thing when your operating system is right there? You can make a file called "orders" and you can run your agent as a user with write access to that file, or as one that doesn't, and then you don't need scans or audits to tell you whether the agent can create orders or not, you can just take your operating system's word for it. Is there anything all this bolt-on AI securi…

The user separation isn’t even necessary, as far as I’ve seen in the projects in https://github.com/bureado/awesome-agent-runtime-security

[dead]

Re: Show HN: Scan your AI agents for dangerous capabilities

#19

Post by smashini »

quixoticaxolotl wrote: post #13
__MatrixMan__ wrote: post #3

Why build separate frameworks for this kind of thing when your operating system is right there? You can make a file called "orders" and you can run your agent as a user with write access to that file, or as one that doesn't, and then you don't need scans or audits to tell you whether the agent can create orders or not, you can just take your operating system's word for it. Is there anything all this bolt-on AI securi…

One benefit is that this can run in serverless / sandboxed containers where OS primitives are not exposed or heavily limited. I immediately thought of Cloudflare Workers, which runs on V8 and exposes WASM-only interfaces, using Workers AI. Further, servers still have hosting value, but any business running agents is almost certainly going to want a sandbox that limits what code runs for agentic work, so targeting _sa…

Completely agree, though the LLM part of the scanner can help with that contextual part of the analysis.

Runtime enforcement already exists (the embedded governor wraps tool calls in-process); extending it with a quarantined contextual evaluator like you describe would the logical next step.

Thanks for the feedback, actually will raise an issue on that to explore

Re: Show HN: Scan your AI agents for dangerous capabilities

#20

Post by smashini »

seethishat wrote: post #15
smashini wrote: post #8

Earlier quoted context omitted.

So Linux can prevent an agent from opening /etc/passwd. Linux cannot stop an agent from calling: POST /wire-transfer amount=5,000,000

Any LSM can stop that POST. SELinux, AppArmor, Tomoyo, etc. They are built-in to the kernel. You just need to know what you are doing to use them.

Yes, but they operate at the OS boundary, but we are defining application/business logic

OS controls answer "can this process make network requests?"

App-level policies answer "is this request, with these parameters, acceptable?"

Post reply on HN