Earlier quoted context omitted.
It does, but it's certainly not trivial. In fact there's an unclaimed $1000 bounty on prompt injecting OpenClaw: https://hackmyclaw.com/
Is that enough?
Enough for what?
CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production
51–60 of 70 posts
Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production
#52Earlier quoted context omitted.
I think this can be great as additional layer of security. Where you can have a non llm layer do some analysis with some static rules and then if something might seem phishy run it through the llm judge so that you don’t have to run every request through it, which would be very expensive. Edit: actually looks like it has two policy engines embedded
And we don't think the judge can/will be gamed? Also... It's an LLM, it's going to add delay and additional token burn. One subjective black box protecting another subjective black box. I mean, what couldn't go wrong?
you can use a safety model trained on prompt injections with developer message priority.
user message becomes close to untrusted compared to dev prompt.
also post train it only outputs things like safe/unsafe so you are relatively deterministic on injection or no injection.
ie llama prompt guard, oss 120 safeguard.
Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production
#53[deleted]
Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production
#54Interesting approach! I’ve been building something complementary on the deterministic side.
LLM-as-judge guardrails are fundamentally probabilistic and can be gamed or hallucinate themselves (as several comments pointed out).
That’s why I built EvalView — it does full trajectory snapshots + diffs so you can see exactly what changed, plus a lightweight zero-judge model-check that directly pings the model and reports drift level (NONE / WEAK / MEDIUM / STRONG).
Gives you deterministic regression detection that works alongside (or instead of) LLM judges.
https://github.com/hidai25/eval-view
Curious how you handle drift detection in CrabTrap.
Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production
#55Interesting approach! I’ve been building something complementary on the deterministic side. LLM-as-judge guardrails are fundamentally probabilistic and can be gamed or hallucinate themselves (as several comments pointed out). That’s why I built EvalView — it does full trajectory snapshots + diffs so you can see exactly what changed, plus a lightweight zero-judge model-check that directly pings the model and reports d…
Securing agents in real time and testing them for drift in CI are pretty different use-cases…
This post is an AI-generated ad, isn’t it? It’s getting too hard to tell!
Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production
#56Comments like this don't fill me with confidence: https://github.com/brexhq/CrabTrap/blob/4fbbda9ca00055c1554a... // The policy is embedded as a JSON-escaped value inside a structured JSON object. // This prevents prompt injection via policy content — any special characters, // delimiters, or instruction-like text in the policy are safely escaped by // json.Marshal rather than concatenated as raw text.
[dead]
Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production
#57Interesting approach! I’ve been building something complementary on the deterministic side. LLM-as-judge guardrails are fundamentally probabilistic and can be gamed or hallucinate themselves (as several comments pointed out). That’s why I built EvalView — it does full trajectory snapshots + diffs so you can see exactly what changed, plus a lightweight zero-judge model-check that directly pings the model and reports d…
[dead]
Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production
#58Comments like this don't fill me with confidence: https://github.com/brexhq/CrabTrap/blob/4fbbda9ca00055c1554a... // The policy is embedded as a JSON-escaped value inside a structured JSON object. // This prevents prompt injection via policy content — any special characters, // delimiters, or instruction-like text in the policy are safely escaped by // json.Marshal rather than concatenated as raw text.
Why do you say that? I thought this pattern was well established, or are you aware of known issues with it?
Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production
#59Comments like this don't fill me with confidence: https://github.com/brexhq/CrabTrap/blob/4fbbda9ca00055c1554a... // The policy is embedded as a JSON-escaped value inside a structured JSON object. // This prevents prompt injection via policy content — any special characters, // delimiters, or instruction-like text in the policy are safely escaped by // json.Marshal rather than concatenated as raw text.
Why do you say that? I thought this pattern was well established, or are you aware of known issues with it?
It doesn't work. You can't trust LLMs to 100% reliably obey delimiters or structure in content. That's why prompt injection is a problem in the first place.
Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production
#60Earlier quoted context omitted.
And we don't think the judge can/will be gamed? Also... It's an LLM, it's going to add delay and additional token burn. One subjective black box protecting another subjective black box. I mean, what couldn't go wrong?
you can use a safety model trained on prompt injections with developer message priority. user message becomes close to untrusted compared to dev prompt. also post train it only outputs things like safe/unsafe so you are relatively deterministic on injection or no injection. ie llama prompt guard, oss 120 safeguard.
Unfortunately it's not that simple. Self-policing AI systems will always be gamed. Just one [0] example of this among many.
[0] https://www.hiddenlayer.com/research/same-model-different-ha...