Live data from Hacker News

Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

github.com

41–50 of 62 posts

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#41
This looks great. I have a use case for something similar: detecting calls to the file system. Lots of code I've inherited has a habit of loading configuration from some random network share, then failing when that config got moved or the production host doesn't have the same access.

I usually use strace(1) to track these down, but it's nowhere near as ergonomic as this tool. I'm wondering now if I could patch the `open` built-in instead.

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#42
post #41

This looks great. I have a use case for something similar: detecting calls to the file system. Lots of code I've inherited has a habit of loading configuration from some random network share, then failing when that config got moved or the production host doesn't have the same access. I usually use strace(1) to track these down, but it's nowhere near as ergonomic as this tool. I'm wondering now if I could patch the `o…

Went spelunking through the source. I think you absolutely could!

There's actually not a whole lot I found that's really http-library specific. It uses the traceback module in a decorator that ends up being manually wrapped around all of the functions of the specific libraries the author cared about.

https://github.com/cle-b/httpdbg/blob/main/httpdbg/hooks

Should be easy enough to extend this to other libraries.

Super cool tool thanks for sharing @dmurray!

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#44
post #41

This looks great. I have a use case for something similar: detecting calls to the file system. Lots of code I've inherited has a habit of loading configuration from some random network share, then failing when that config got moved or the production host doesn't have the same access. I usually use strace(1) to track these down, but it's nowhere near as ergonomic as this tool. I'm wondering now if I could patch the `o…

If on Linux or windows you can use Procmon or Instruments on macos.

https://github.com/Sysinternals/ProcMon-for-Linux

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#45
post #29

I've always used a proxy, like charles proxy, for this exact purpose. A neutral middle-man that gives exact timing/response data.

I'm really surprised this was downvoted. Running e.g. mitmproxy and pointing your Python process at it is absolutely the way to do this.

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#46
Can recommend Opentelemetry if you need a more comprehensive tool like this.

There is a whole library of so called instrumentation that can monkeypatch standard functions and produce traces of them.

Traces can also propagate across process and rpc, giving you a complete picture, even in a microservice architecture.

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#47

Earlier quoted context omitted.

Here's a concrete scenario for you: Say you are in a team of 10 developers with a huge codebase that has accumulated over 5+ years. If you're new in the team, and you need to understand when a specific HTTP header is sent, or just snoop the value in the payload you otherwise wouldn't be able to see.

Snooping traffic isn't new though, so what's specific about this tool and Python.

How would you snoop outgoing HTTPS traffic otherwise easily anyway? mitmproxy requires some work to set up

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#48
post #41

This looks great. I have a use case for something similar: detecting calls to the file system. Lots of code I've inherited has a habit of loading configuration from some random network share, then failing when that config got moved or the production host doesn't have the same access. I usually use strace(1) to track these down, but it's nowhere near as ergonomic as this tool. I'm wondering now if I could patch the `o…

You might find the syscall tracing functionality of Cirron useful: https://github.com/s7nfo/Cirron

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#49
post #29

I've always used a proxy, like charles proxy, for this exact purpose. A neutral middle-man that gives exact timing/response data.

It's not as simple as it sounds, it requires a lot of code to capture Python traffic with charles-proxy. For example, you might modify your python code to use a Proxy and accept a self-signed Charles's certificate.

If you need a 1-click solution, no dependencies, and no code's required, check out Proxyman with Auto-Setup: https://docs.proxyman.io/automatic-setup/automatic-setup

Works with all popular Python libs: request, aiohttp, http.client, urllib3, etc

* Disclaimer: I'm Noah, creator of Proxyman. I know a pain point when using Charles, and decided to build a new one, to make life easier. Hope it helps you.

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#50
post #46

Can recommend Opentelemetry if you need a more comprehensive tool like this. There is a whole library of so called instrumentation that can monkeypatch standard functions and produce traces of them. Traces can also propagate across process and rpc, giving you a complete picture, even in a microservice architecture.

Is there an example?
Post reply on HN