Live data from Hacker News

Httptap: View HTTP/HTTPS requests made by any Linux program

github.com

81–90 of 148 posts

Re: Httptap: View HTTP/HTTPS requests made by any Linux program

#81
post #65

That's a great DX! I wonder if an alternative way is to just hook functions like read, and write, and other functions linked to the SSL libs. It is true that you should be aware of the SSL libs in place but OpenSSL is the most popular.

This is a very interesting possibility actually. Not 100% sure what the implication would be but maybe there is something here.

You can use a hooking library like these [1].

[1] https://github.com/frida/frida/wiki/Comparison-of-function-h...

Re: Httptap: View HTTP/HTTPS requests made by any Linux program

#83

Earlier quoted context omitted.

Agreed! So there isn't any interface for modifying requests/responses at present, but it's definitely possible given the underlying approach. If you consider [this line of code]( https://github.com/monasticacademy/httptap/blob/main/http.go... ) where you have an HTTP request parsed from the that ran and are about to send it out to the public internet: you could modify the request (or the response that is received a f…

Injecting random data into telemetry requests to mess up someone’s pretty dashboard?

There's at least https://adnauseam.io/ for that already

Re: Httptap: View HTTP/HTTPS requests made by any Linux program

#84
post #25

Earlier quoted context omitted.

You might find mitmproxy useful.

Yep, mitmproxy is fantastic IMO.

Apparently there's JS interface for it too: https://www.npmjs.com/package/mitmproxy

But the problem with it is that you have to configure proxy in the app (they do got transparent and wireguard modes but there's still quite a bit of configuration).

Re: Httptap: View HTTP/HTTPS requests made by any Linux program

#85

Earlier quoted context omitted.

Thanks! Haven't measured performance at all. However when I decided on the approach I looked at the performance benchmarks from tun2socks, which uses the same gVisor code that httptap uses, and it seems that pretty great performance is possible with that code. Still need to do the work of actually achieving comparable performance. Payloads and headers can be dumped with --body and --header respectively. There is an e…

I see it supports .har, but .warc support would be amazing. It's the iso standard for web archives, which would give you the ability to replay archived websites via multiple tools.

Fascinating, I hadn't heard of WARC - thanks for the pointer and I'll consider it.

Re: Httptap: View HTTP/HTTPS requests made by any Linux program

#86

Why not use eBPF instead? Then you could see all http requests from all processes at once, including ones that are already running. Plus you wouldn't need to bother with TLS at all, just hook on e.g. write(2).

Unfortunately TLS happens inside the the application, not in the kernel, so using eBPF to hook syscalls to write won't help with TLS decryption.

Re: Httptap: View HTTP/HTTPS requests made by any Linux program

#87

Which privileges are required? CAP_NET_ADMIN? Or nothing at all?

Nothing at all!

You do need write access to /dev/net/tun. This is standard for all users for the distros that I've looked into, but it is ultimately a distro-specific thing.

Re: Httptap: View HTTP/HTTPS requests made by any Linux program

#88
post #79

This is really cool, thank you for sharing! We've built a similar feature for mitmproxy lately, but with different tradeoffs. Our approach does require root and we don't have automated certificate install (yet), but we don't require apps to run in a dedicated namespace (so you can capture already-running processes). Super awesome to see this now, excited to dive into the code and see how you do TCP reassembly etc. :)

Thank you! mitmproxy is fantastic - thanks for all the work that's gone into that project. Maybe we can get in touch and chat about all this stuff.

Re: Httptap: View HTTP/HTTPS requests made by any Linux program

#89

Earlier quoted context omitted.

Inside most production environments. I could use this today inside a Pod that isn't allowed root privs.

In production environments that won't give you root access, you won't be exec'ing inside of a pod if you aren't an operator or sysadmin.

In my particular case, I am an operator and sys admin, but I don't give myself root privileges without having to go through some serious hoops, which I only jump through if I really truly need it. If I want root, I have to actually change the kubernetes manifest yaml to allow elevation to root privileges. That's not something that can be done without getting others involved for code reviews and what not.

However, even in the case of general developers, it isn't true. Companies do restrict exec abilities, but we don't. Many startups are the same, because developers are expected to also troubleshoot and debug production issues. If you don't allow shells in pods, you are really binding the hands of your devs.

To be clear, I am not disagreeing with you. You are correct in many cases. But there are a number of exceptions in my experience.

Re: Httptap: View HTTP/HTTPS requests made by any Linux program

#90

Why not use eBPF instead? Then you could see all http requests from all processes at once, including ones that are already running. Plus you wouldn't need to bother with TLS at all, just hook on e.g. write(2).

Unfortunately TLS happens inside the the application, not in the kernel, so using eBPF to hook syscalls to write won't help with TLS decryption.

It is quite simple to use eBPF with uprobes to hook library calls, for example: https://github.com/iovisor/bcc/blob/master/tools/sslsniff.py

The downside is this doesn't work with anything not using OpenSSL, there are projects like https://github.com/gojue/ecapture which have interceptors for many common libraries, but the downside is that needs different code for each library.

I think providing a TLS certificate is fine for the use cases of the tool; most tools won't be doing certificate pinning, but ecapture does support Android where this is more likely.

Post reply on HN