Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

brex.com

Post reply on HN
Print view View original

11–20 of 70 posts

Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

#11

Post by nl »

DANmode wrote: post #3

We’re supposed to be fixing LLM security by adding a non-LLM layer to it, not adding LLM layers to stuff to make them inherently less secure. This will be a neat concept for the types of tools that come after the present iteration of LLMs. Unless I’m sorely mistaken.

> We’re supposed to be fixing LLM security by adding a non-LLM layer to it,

If people said "we build a ML-based classifier into our proxy to block dangerous requests" would it be better? Why does the fact the classifier is a LLM make it somehow worse?

Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

#12

Post by vova_hn2 »

ImPostingOnHN wrote: post #9
snug wrote: post #7

Earlier quoted context omitted.

I think this can be great as additional layer of security. Where you can have a non llm layer do some analysis with some static rules and then if something might seem phishy run it through the llm judge so that you don’t have to run every request through it, which would be very expensive. Edit: actually looks like it has two policy engines embedded

What happens when a prompt injection attack exploits the judge LLM and results in a higher level of attacker control than if it never existed?

How can it result in a higher level of control? I don't see why the "judge" should have access to anything except one tool that allows it to send an "accept" or "deny" command.

Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

#13

Post by Retr0id »

nl wrote: post #11
DANmode wrote: post #3

We’re supposed to be fixing LLM security by adding a non-LLM layer to it, not adding LLM layers to stuff to make them inherently less secure. This will be a neat concept for the types of tools that come after the present iteration of LLMs. Unless I’m sorely mistaken.

> We’re supposed to be fixing LLM security by adding a non-LLM layer to it, If people said "we build a ML-based classifier into our proxy to block dangerous requests" would it be better? Why does the fact the classifier is a LLM make it somehow worse?

The fact that LLMs are "smarter" is also their weakness. An oldschool classifier is far from foolproof, but you won't get past it by telling it about your grandma's bedtime story routine.

Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

#15

Post by waterTanuki »

nl wrote: post #11
DANmode wrote: post #3

We’re supposed to be fixing LLM security by adding a non-LLM layer to it, not adding LLM layers to stuff to make them inherently less secure. This will be a neat concept for the types of tools that come after the present iteration of LLMs. Unless I’m sorely mistaken.

> We’re supposed to be fixing LLM security by adding a non-LLM layer to it, If people said "we build a ML-based classifier into our proxy to block dangerous requests" would it be better? Why does the fact the classifier is a LLM make it somehow worse?

If you're working in a mission-critical field like healthcare, defense, etc. you need a way to make static and verifiable guarantees that you can't leak patient data, fighter jet details etc. through your software. This is either mandated by law or in your contract details.

The entire purpose of LLMs is to be non-static: they have no deterministic output and can't be validated the same way a non-LLM function can be. Adding another LLM layer is just adding another layer of swiss cheese and praying the holes don't line up. You have no way of predicting ahead of time whether or not they will.

You might say this hasn't prevented leaks/CVEs in exisiting mission-critical software and this would be correct. However, the people writing the checks do not care. You get paid as long as you follow the spec provided. How then, in a world which demands rigorous proof do you fit in an LLM judge?

Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

#20

Post by stingraycharles »

reassess_blind wrote: post #5
DANmode wrote: post #3

We’re supposed to be fixing LLM security by adding a non-LLM layer to it, not adding LLM layers to stuff to make them inherently less secure. This will be a neat concept for the types of tools that come after the present iteration of LLMs. Unless I’m sorely mistaken.

It looks as if this tool has traditional static rules to allow/deny requests, as well as a secondary LLM-as-a-judge layer for, I imagine, the kinds of rules that would be messy or too convoluted to implement using standard rules.

I think the parent’s point is that this should be implemented using e.g. Bayesian statistics rather than an LLM, as the judge LLM is vulnerable to the exact same types of attacks that it’s trying to protect against.

Most proper LLM guardrails products use both.

Post reply on HN