Live data from Hacker News

Httptap: View HTTP/HTTPS requests made by any Linux program

github.com

61–70 of 148 posts

Re: Httptap: View HTTP/HTTPS requests made by any Linux program

#61

Is this implementing TCP in userspace?

Yep. This is the first time I've mentioned this but there are actually two implementations of this in the codebase -- one uses gVisor and one is an incredibly bare-bones TCP implementation that I wrote myself in 550 lines of Go code (tcp.go). The home-grown one isn't used by default and it doesn't support much of TCP proper, but it actually works pretty well. You can use it with `--stack=homegrown`.

Re: Httptap: View HTTP/HTTPS requests made by any Linux program

#62
post #15

This is something I’ve needed more than a handful of times - and each time slowly figured out a cobbled together solution with wireshark / pcap Fun reading about the authors too. Thank you for sharing !

Same here actually -- have wanted this from time to time for years. Finally sat down and worked it out.

Re: Httptap: View HTTP/HTTPS requests made by any Linux program

#63

Does this work with larger more complicated software like web browsers, skype, or discord? I know I'd have to run firefox with --no-remote. Very cool idea though, love tools with this sort of UX. I look forward to a V1 release in the future.

Thanks!

I did try this with firefox but it doesn't work right now due to (I think) the user namespace messing with user IDs. I think I should be able to fix this, though. I will have to try it with other desktop apps soon too...

Re: Httptap: View HTTP/HTTPS requests made by any Linux program

#64
post #13
post #4

The "How it was made" section of the README was not less interesting than the tool itself: > The way we have set things up is that we live and practice together on a bit over a hundred acres of land. In the mornings and evenings we chant and meditate together, and for about one week out of every month we run and participate in a meditation retreat. The rest of the time we work together on everything from caring for t…

Reminds me of a quote from "Soul of a new machine": > During one period, when the microcode and logic were glitching at the nanosecond level, one of the overworked engineers departed the company, leaving behind a note on his terminal as his letter of resignation: "I am going to a commune in Vermont and will deal with no unit of time shorter than a season."

Great quote, although the nitpicky part of my brain immediately thought "They must have days though?"

Re: Httptap: View HTTP/HTTPS requests made by any Linux program

#66
post #43

Mitmproxy v11.1 can do a similar thing

Yeah mitmproxy is great. The main difference with httptap is that it's an HTTP proxy server, so you have to configure your program to use a proxy server. When I wrote httptap I wanted to be able to run `httptap ` and see the httptraces right there in standard output. There is an absolute ton of cool things that mitmproxy can do that httptap is not even close to, like interactively modifying HTTP requests and such. Very cool project.

Re: Httptap: View HTTP/HTTPS requests made by any Linux program

#67
post #65

That's a great DX! I wonder if an alternative way is to just hook functions like read, and write, and other functions linked to the SSL libs. It is true that you should be aware of the SSL libs in place but OpenSSL is the most popular.

This is a very interesting possibility actually. Not 100% sure what the implication would be but maybe there is something here.

Re: Httptap: View HTTP/HTTPS requests made by any Linux program

#68
post #54
post #41

Earlier quoted context omitted.

Yes, good point, maybe that is the blocker.

No network namespaces, but the various Network Extension APIs might be able to do this, though it's difficult. RocketSim (I'm unaffiliated, just an example) recently added a simulator-specific network throttle (to replace the system-wide Network Link Conditioner Apple ships) using a content filter extension. Even though this is a system-wide API, it seems you can limit its impact to a single app. And it seemed to pro…

Woah, this is super helpful info. Thanks. That sounds like a real possibility for a macOS port actually.

Re: Httptap: View HTTP/HTTPS requests made by any Linux program

#69

httptap is a process-scoped http tracer that you can run without root priveleges. You can run `httptap ` where is a linux program and you get a trace of http/https requests and responses in standard output: httptap -- python -c "import requests; requests.get('https://monasticacademy.org')" ---> GET https://monasticacademy.org/ GET https://www.monasticacademy.org/ It works by running in an isolated network namespace.…

Did everyone forget about wireshark, which can totally be ran as non-root? https://blog.wireshark.org/2010/02/running-wireshark-as-you/

Wireshark is awesome but yeah as others mentioned it's the TLS decryption piece that is difficult in that workflow

Re: Httptap: View HTTP/HTTPS requests made by any Linux program

#70
post #44
post #4

The "How it was made" section of the README was not less interesting than the tool itself: > The way we have set things up is that we live and practice together on a bit over a hundred acres of land. In the mornings and evenings we chant and meditate together, and for about one week out of every month we run and participate in a meditation retreat. The rest of the time we work together on everything from caring for t…

I sadly assumed the first countryside photo was generated but I assume now it is real! The mix of tech and meditation would appeal to me. Maybe the idea does (actually doing it is probably hard!). It seems like a "Buddhist Recurse"

Yeah that photo is real! That's where I live!

Yes, it's true, actually doing it is hard, but to be honest not as hard as a lot of other stuff (getting a phd for example, or goodness gracious buying a house in San Francisco). I love getting up early. I love living out in nature. I love chanting and eating meals together and making a version of Buddhism for AI systems!

If you're interested in what it's like, we have written a bunch of very short few-paragraph stories about our time at MAPLE here: https://tales.monasticacademy.org/

Post reply on HN