Live data from Hacker News

CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

brex.com

41–50 of 70 posts

Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

#41
post #32

[flagged]

I'm willing to wager that your comment was generated from the body of the article plus a prompt to work in an advertisement for your product, which gets a mention in nearly every comment you make (and every submission you make, sometimes on a daily basis).

Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

#44
post #28

[flagged]

> Why it lands: specific technical question, credits their work, ends with something that invites response. If Brex engineers are in the thread, one of them will likely reply.

BWHAHAHAHAHA. your bot tried, but failed at the same time. (also interesting that this user's other comments seem ok-ish. The prompts are evolving, we get a sneak peek here on what they prompted for, and the delivery seems more human as well)

Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

#47
post #32

[flagged]

I'm willing to wager that your comment was generated from the body of the article plus a prompt to work in an advertisement for your product, which gets a mention in nearly every comment you make (and every submission you make, sometimes on a daily basis).

Hand written I’m afraid… regular comments on this topic is true - it’s an area I’m very interested in.

Re: CrabTrap: An LLM-as-a-judge HTTP proxy to secure agents in production

#49

> pointing it at a few days of real traffic produced policies that matched human judgment on the vast majority of held-out requests. The problem is, 99% secure is a failing grade.

99% is usually the best you can do. So you can only layer multiple defences together, this makes sense as one layer to me.

I have an issue with security layers that are inherently nondeterministic. You can't really reason strongly about what this tool provides as part of a security model.

But also, it's in an area where real security seems extremely hard. I think at some point everyone will have a situation where they wanna give an agent some private information and access to the web. You just can't do that in a way that's deterministically safe. But if there are usecase where making it probabilistically safer is enough to tip the balance, well, fine.

Post reply on HN