Live data from Hacker News

Show HN: A MitM proxy to see what your LLM tools are sending

github.com

51–60 of 128 posts

Re: Show HN: A MitM proxy to see what your LLM tools are sending

#51

Nice work! I'm sure the data gleaned here is illuminating for many users. I'm surprised that there isn't a stronger demand for enterprise-wide tools like this. Yes, there are a few solutions, but when you contrast the new standard of "give everyone at the company agentic AI capabilities" with the prior paradigm of strong data governance (at least at larger orgs), it's a stark difference. I think we're not far from th…

I had to vibe code a proxy to hide tokens from agents ( https://github.com/vladimirkras/prxlocal ) because I haven’t found any good solution either. I planned to add genai otel stuff that could be piped into some tool to view dialogues and tool calls and so on, but I haven’t found any good setup that doesn’t require lots of manual coding yet. It’s really weird that there are no solutions in that space.

nice, I'm working on something similar with macroons so the tokens can be arbitrarily scopes in time and capability too.

Mine uses an Envoy sidecar on a sandbox container.

https://github.com/dtkav/agent-creds

Re: Show HN: A MitM proxy to see what your LLM tools are sending

#52
post #50
post #46

Earlier quoted context omitted.

Yeah would love this for logfire

Something like sherlock start --otel-endpoint?

Yes. It can get a bit more complex as some otels require authentication. You can check Pydantic AI Gateway, Cloudflare AI Gateway or LiteLLM itself. They do similar things. One advantage of yours would be simplicity.

Re: Show HN: A MitM proxy to see what your LLM tools are sending

#53
post #40

Earlier quoted context omitted.

Addon to mitmproxy: https://docs.mitmproxy.org/stable/addons/overview/

What would you think of simply using an http relay for all providers? Would that make you feel better secutity wise? also could extend the tool to change the context you are sending and make it more granular to what you want/need...

[deleted]

Re: Show HN: A MitM proxy to see what your LLM tools are sending

#54
This tool looks like it unconditionally disables tls verification for upstream requests.

It shells out to mitmproxy with "--set", "ssl_insecure=true"

This took all of 5 minutes to find reading through main.py on my phone.

https://github.com/jmuncor/sherlock/blob/fb76605fabbda351828...

Edit: In case it’s not clear, you should not use this.

Re: Show HN: A MitM proxy to see what your LLM tools are sending

#55
post #32

Earlier quoted context omitted.

Agee! This was a fun project that I build because it is so hard to understand what "really" is in you context window... What do you mean by plugin/add-on? Add-on to what? Thinking of what to add to it next... Maybe security would be a good direction, or at least visibility of what is happening to the proxy's traffic.

Addon to mitmproxy: https://docs.mitmproxy.org/stable/addons/overview/

Looks like it's implemented that way already?

https://github.com/jmuncor/sherlock/blob/fb76605fabbda351828...

Re: Show HN: A MitM proxy to see what your LLM tools are sending

#57

Could you use an approach like this much like a traditional network proxy, to block or sanitise some requests? E.g. if a request contains confidential information (whatever you define that to be), then block it?

Forgot to mention: It’s a neat tool. Well done.
Post reply on HN