Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

brex.com

Post reply on HN
Print view View original

31–40 of 70 posts

Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

#31

Post by nl »

waterTanuki wrote: post #15
nl wrote: post #11

Earlier quoted context omitted.

> We’re supposed to be fixing LLM security by adding a non-LLM layer to it, If people said "we build a ML-based classifier into our proxy to block dangerous requests" would it be better? Why does the fact the classifier is a LLM make it somehow worse?

If you're working in a mission-critical field like healthcare, defense, etc. you need a way to make static and verifiable guarantees that you can't leak patient data, fighter jet details etc. through your software. This is either mandated by law or in your contract details. The entire purpose of LLMs is to be non-static: they have no deterministic output and can't be validated the same way a non-LLM function can be.…

> The entire purpose of LLMs is to be non-static: they have no deterministic output and can't be validated the same way a non-LLM function can be. Adding another LLM layer is just adding another layer of swiss cheese and praying the holes don't line up. You have no way of predicting ahead of time whether or not they will.

This is exactly the point though. A LLM is great at finding work-around for static defenses. We need something that understands the intent and responds to that.

Static rules are insufficient

Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

#34

Post by DANmode »

reassess_blind wrote: post #27
Retr0id wrote: post #24

Earlier quoted context omitted.

That specific trick yes, but the general concept still applies.

It does, but it's certainly not trivial. In fact there's an unclaimed $1000 bounty on prompt injecting OpenClaw: https://hackmyclaw.com/

Is that enough?

Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

#35

Post by lmeyerov »

edf13 wrote: post #32

[flagged]

At RSAC, there were a ton of agentic security startups converging on ebpf monitors for this reason. Eg, sondera gave a fun talk at graph the planet where they did that + exposed with a policy layer over agent traces via Cedar (used in AWS IAM etc). ABAC and identity were also appearing near here.

One thing I didn't see: are there any OSS solutions appearing here?

Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

#36

Post by edf13 »

lmeyerov wrote: post #35
edf13 wrote: post #32

[flagged]

At RSAC, there were a ton of agentic security startups converging on ebpf monitors for this reason. Eg, sondera gave a fun talk at graph the planet where they did that + exposed with a policy layer over agent traces via Cedar (used in AWS IAM etc). ABAC and identity were also appearing near here. One thing I didn't see: are there any OSS solutions appearing here?

We are Open Source… code will be published soon (before launch)

Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

#37

Post by NamlchakKhandro »

reassess_blind wrote: post #22
fc417fc802 wrote: post #18

Earlier quoted context omitted.

Calling it now. Show HN: Pincer - A small highly optimized local model to detect prompt injection attempts against other models.

Sounds like a good idea. Please send me the Github link once done and I'll have my OpenClaw take a look and form my opinion of it.

Sounds like a good idea. Please send me you GitHub now and I'll have my big claw crush your open claw

Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

#39

Post by lanyard-textile »

edf13 wrote: post #36
lmeyerov wrote: post #35

Earlier quoted context omitted.

At RSAC, there were a ton of agentic security startups converging on ebpf monitors for this reason. Eg, sondera gave a fun talk at graph the planet where they did that + exposed with a policy layer over agent traces via Cedar (used in AWS IAM etc). ABAC and identity were also appearing near here. One thing I didn't see: are there any OSS solutions appearing here?

We are Open Source… code will be published soon (before launch)

Then you will be open source ;) Not yet open source.

Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

#40

Post by manapause »

yakkomajuri wrote: post #2

Really cool! I'm also building something in this space but taking a slightly different approach. I'm glad to see more focus on security for production agentic workflows though, as I think we don't talk about it enough when it comes to claws and other autonomous agents. I think you're spot on with the fact that it's so far it's been either all or nothing. You either give an agent a lot of access and it's really powerf…

Correct me if I’m wrong, but from my experience in this space in order for a model to exercise judgment it must force itself to operate in a strict chain of thought mode. Since all LLMs are predictive creatures, I started to care a lot more about my judgment settings, the transparency of them, and the presence of a judgment loop in either the development or functionality of an application built these days.

Not exactly sure where I’m going with this, but my work with creating penetesting tools for LLMs, the way that I use judgment is critical to the core functionality of the application. I agree with your concern and I will just say that the more time I spent concerned with chain of though where now I will make multiple versions of the same app using a different judge set a different “temperaments” and I found it to be incredibly enlightening as to the diversity of applications and approaches that it creates.

  Even using BMAD or superpowers, I can make five versions of an app without judges involved and I feel like I’m just making the same app five times because the API begins to coalesce around the business problem you want to solve. The vicissitudes of prediction tools always want to take the safest bet for the greater good, but with the judge involved we can make the agent force itself to actually be hostile about what exactly we’re trying to do, which has produced interesting and fun results.
Post reply on HN