Live data from Hacker News

What I learned by putting GitHub Copilot behind a MitM proxy

lighthousenewsletter.com

21–30 of 40 posts

Re: What I learned by putting GitHub Copilot behind a MitM proxy

#21

I believe this can be done without a MitM by using Wireshark or adjacent tool

It's possible, but not easy. You need to export the internal state of your TLS stack to Wireshark in order for it to decrypt the traffic. This can be done via LD_PRELOAD type shenanigans but it's difficult and fragile.

MITM proxy is much easier to get working.

Re: What I learned by putting GitHub Copilot behind a MitM proxy

#22
Disagree with the conclusion, even without carefully curated context every high end LLM perform just as well, maybe with an extra detour. In contrast if even one of the learnings is not up to date or doesn't apply to the current situation you find yourself with a long detour or even a failure.

Re: What I learned by putting GitHub Copilot behind a MitM proxy

#24
post #3

I was curious to understand how Copilot implements its harness, and also how I was exhausting my quota so quickly. End up going down a rabbit hole of intercepting its network traffic with mitmproxy. A few interesting things I found along the way: - watched model/capability discovery and routing happen in real time - looked at what gets injected into context and sent with ghost completions - found that recent edits ca…

How do you actually cleanly solve that .env issue?

Anything cross platform and coding agent agnostic?

I suppose that .env file should be removed, but then things aren’t easy: no native multiplatform secret manager, or the std lib of the language doesn’t offer an API over the native secret store, etc.

Or a "secret injection proxy" for some cases could work I guess.

Re: What I learned by putting GitHub Copilot behind a MitM proxy

#25
post #24
post #3

I was curious to understand how Copilot implements its harness, and also how I was exhausting my quota so quickly. End up going down a rabbit hole of intercepting its network traffic with mitmproxy. A few interesting things I found along the way: - watched model/capability discovery and routing happen in real time - looked at what gets injected into context and sent with ghost completions - found that recent edits ca…

How do you actually cleanly solve that .env issue? Anything cross platform and coding agent agnostic? I suppose that .env file should be removed, but then things aren’t easy: no native multiplatform secret manager, or the std lib of the language doesn’t offer an API over the native secret store, etc. Or a "secret injection proxy" for some cases could work I guess.

Data retention clauses?

I dont see how you can ever really trust an LLM anyway to follow instructions.

Re: What I learned by putting GitHub Copilot behind a MitM proxy

#26
post #24
post #3

I was curious to understand how Copilot implements its harness, and also how I was exhausting my quota so quickly. End up going down a rabbit hole of intercepting its network traffic with mitmproxy. A few interesting things I found along the way: - watched model/capability discovery and routing happen in real time - looked at what gets injected into context and sent with ghost completions - found that recent edits ca…

How do you actually cleanly solve that .env issue? Anything cross platform and coding agent agnostic? I suppose that .env file should be removed, but then things aren’t easy: no native multiplatform secret manager, or the std lib of the language doesn’t offer an API over the native secret store, etc. Or a "secret injection proxy" for some cases could work I guess.

Do you feel that something like https://secretspec.dev/ addresses these points?

Re: What I learned by putting GitHub Copilot behind a MitM proxy

#27
post #24
post #3

I was curious to understand how Copilot implements its harness, and also how I was exhausting my quota so quickly. End up going down a rabbit hole of intercepting its network traffic with mitmproxy. A few interesting things I found along the way: - watched model/capability discovery and routing happen in real time - looked at what gets injected into context and sent with ghost completions - found that recent edits ca…

How do you actually cleanly solve that .env issue? Anything cross platform and coding agent agnostic? I suppose that .env file should be removed, but then things aren’t easy: no native multiplatform secret manager, or the std lib of the language doesn’t offer an API over the native secret store, etc. Or a "secret injection proxy" for some cases could work I guess.

Infisical, or Bitwarden Secret Manager? Those two look like perfectly reasonable if the llm is just careless (but still, nothing prevents the LLM from intentionally cat'ing /proc/self/environ or from running /usr/bin/env or set or similar)

Re: What I learned by putting GitHub Copilot behind a MitM proxy

#28
post #24
post #3

I was curious to understand how Copilot implements its harness, and also how I was exhausting my quota so quickly. End up going down a rabbit hole of intercepting its network traffic with mitmproxy. A few interesting things I found along the way: - watched model/capability discovery and routing happen in real time - looked at what gets injected into context and sent with ghost completions - found that recent edits ca…

How do you actually cleanly solve that .env issue? Anything cross platform and coding agent agnostic? I suppose that .env file should be removed, but then things aren’t easy: no native multiplatform secret manager, or the std lib of the language doesn’t offer an API over the native secret store, etc. Or a "secret injection proxy" for some cases could work I guess.

[flagged]

Re: What I learned by putting GitHub Copilot behind a MitM proxy

#29
post #12
post #9

One thing I found that I thought was a fun addition, is using eBPF made this even easier. No need to fight with anyone that is using certificate pinning, mTLS or anything else, you just get the raw plaintext data straight of the wire (right before encryption and right after decryption) and works nicely for most of the agents and IDE's. That will in practice give you everything from telemetry to prompts, and its funny…

Surprised that works, I thought TLS was done entirely in process space. I think I found it, My first thought was some sort of builtin ssl library backdoor, but it looks like you do some ld.preload shenanigans to inject a eBPF monitor. I am not sure exactly what the BPF brings to the table here. A convenient interface to intercept the accept() syscall?

It is similar but no real shenanigans but rather a common functionality called uprobes. For most of these LLM clients which uses a static compiled OpenSSL or BoringSSL the functions you attach to is also the same overall, making it easily reusable when you know what to do. There has been plenty of srticles over the time on it, quite a fun thing to play around with. I made it work already with most of them for research purposes.

Re: What I learned by putting GitHub Copilot behind a MitM proxy

#30
post #16
post #11

Earlier quoted context omitted.

Out of curiosity: How? They don't offload TLS to the kernel, do they? Most apps do it in userspace linked against openssl afaik. Do you patch that lib? If ebpf "just" operates at network/packet level, I don't see how it can do more than Mitmproxy in regard to avoid DH-PFS/Pinning

It’s done via uprobes which allows ebpf to attach to users processes or libraries, basically like ld_preload but built into the kernel. First link that explains it https://blog.quarkslab.com/defeating-ebpf-uprobe-monitoring.... but there are many more links about it and it is a pretty useful tool for debugging in prod environments. (Also first link: https://blog.px.dev/ebpf-function-tracing/ ) this also the intended…

Exactly this :) You can either attach the uprobes directly to the systems OpenSSL to capture anything using it (curl etc), though most of these LLM clients and similar uses their own static compiled BoringSSL, but its very straight forward to find, it does require some knowledge to determine the right place where you attach the uprobe, but after that it can easily be scripted to automatically find the right offsets (as these are usually stripped binaries)
Post reply on HN