Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

github.com

Post reply on HN
Print view View original

11–20 of 62 posts

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#11

Post by robertlagrant »

ricardo81 wrote: post #9

I could be lost here (C/PHP/Node coder mainly in code I've used) Why is it a special case to track HTTP/s requests, that otherwise couldn't be logged like any other process/function? I'd guess most people use libcurl and you can wrap something around that. I guess I'm lost on why this is HTTP or Python specific, or if it is, fine.

I think the nice thing about HTTP for this is different parts of the stack can introduce default headers etc and it's helpful to be able to see the actual request after all that processing's been done.

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#12

Post by seanc »

judofyr wrote: post #10

Looks neat! A similar tool for this would be VCR (originally built in Ruby, but ported to other languages since): https://vcrpy.readthedocs.io/en/latest/ . This injects itself into the request pipeline, records the result in a local file which can then also be replayed later in tests. It's a quite nice approach when you want to write tests (or just explore) a highly complicated HTTP API without actually hitting it al…

The inspection and debugging features this offers are great additions though. I've stared at VCR yaml enough times to not want to ever do it again.

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#13

Post by ricardo81 »

robertlagrant wrote: post #11
ricardo81 wrote: post #9

I could be lost here (C/PHP/Node coder mainly in code I've used) Why is it a special case to track HTTP/s requests, that otherwise couldn't be logged like any other process/function? I'd guess most people use libcurl and you can wrap something around that. I guess I'm lost on why this is HTTP or Python specific, or if it is, fine.

I think the nice thing about HTTP for this is different parts of the stack can introduce default headers etc and it's helpful to be able to see the actual request after all that processing's been done.

With curl there's always CURLOPT_VERBOSE as per the library.

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#14

Post by seanc »

ricardo81 wrote: post #9

I could be lost here (C/PHP/Node coder mainly in code I've used) Why is it a special case to track HTTP/s requests, that otherwise couldn't be logged like any other process/function? I'd guess most people use libcurl and you can wrap something around that. I guess I'm lost on why this is HTTP or Python specific, or if it is, fine.

In the old days we'd use tcpdump and wireshark for this, but nowadays everything is encrypted up in the application layer so you need this kind of thing. Or tricky key dumping hacks.

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#15

Post by cle-b »

ricardo81 wrote: post #9

I could be lost here (C/PHP/Node coder mainly in code I've used) Why is it a special case to track HTTP/s requests, that otherwise couldn't be logged like any other process/function? I'd guess most people use libcurl and you can wrap something around that. I guess I'm lost on why this is HTTP or Python specific, or if it is, fine.

Unlike other tools such as proxies that allow you to trace HTTP requests, httpdbg makes it possible to link the HTTP request to the Python code that initiated it. This is why it is specific to Python and does not work with other languages.

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#16

Post by ricardo81 »

cle-b wrote: post #15
ricardo81 wrote: post #9

I could be lost here (C/PHP/Node coder mainly in code I've used) Why is it a special case to track HTTP/s requests, that otherwise couldn't be logged like any other process/function? I'd guess most people use libcurl and you can wrap something around that. I guess I'm lost on why this is HTTP or Python specific, or if it is, fine.

Unlike other tools such as proxies that allow you to trace HTTP requests, httpdbg makes it possible to link the HTTP request to the Python code that initiated it. This is why it is specific to Python and does not work with other languages.

I'm still not understanding.

If you're coding something up, why wouldn't you know that piece of code does a HTTP/s request? Based on what you said, it sounds like a scenario where a programmer doesn't know how a request was made. Are there examples of scenarios where that's the case?

Sounds like a bit of a security nightmare where there's code doing arbitrary requests.

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#17

Post by cle-b »

judofyr wrote: post #10

Looks neat! A similar tool for this would be VCR (originally built in Ruby, but ported to other languages since): https://vcrpy.readthedocs.io/en/latest/ . This injects itself into the request pipeline, records the result in a local file which can then also be replayed later in tests. It's a quite nice approach when you want to write tests (or just explore) a highly complicated HTTP API without actually hitting it al…

I really like vrcpy. I used it a lot with pytest in my previous job. httpdbg isn’t exactly the same; the idea is more about seeing HTTP requests in real-time and being able to easily study them.

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#18

Post by [deleted] »

judofyr wrote: post #10

Looks neat! A similar tool for this would be VCR (originally built in Ruby, but ported to other languages since): https://vcrpy.readthedocs.io/en/latest/ . This injects itself into the request pipeline, records the result in a local file which can then also be replayed later in tests. It's a quite nice approach when you want to write tests (or just explore) a highly complicated HTTP API without actually hitting it al…

[deleted]

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#19

Post by toomuchtodo »

seanc wrote: post #14
ricardo81 wrote: post #9

I could be lost here (C/PHP/Node coder mainly in code I've used) Why is it a special case to track HTTP/s requests, that otherwise couldn't be logged like any other process/function? I'd guess most people use libcurl and you can wrap something around that. I guess I'm lost on why this is HTTP or Python specific, or if it is, fine.

In the old days we'd use tcpdump and wireshark for this, but nowadays everything is encrypted up in the application layer so you need this kind of thing. Or tricky key dumping hacks.

https://www.charlesproxy.com/ ?

Re: Show HN: Httpdbg – A tool to trace the HTTP requests sent by your Python code

#20

Post by bityard »

ricardo81 wrote: post #16
cle-b wrote: post #15

Earlier quoted context omitted.

Unlike other tools such as proxies that allow you to trace HTTP requests, httpdbg makes it possible to link the HTTP request to the Python code that initiated it. This is why it is specific to Python and does not work with other languages.

I'm still not understanding. If you're coding something up, why wouldn't you know that piece of code does a HTTP/s request? Based on what you said, it sounds like a scenario where a programmer doesn't know how a request was made. Are there examples of scenarios where that's the case? Sounds like a bit of a security nightmare where there's code doing arbitrary requests.

Maybe you are working with an application or library that you didn't write, and want to see the raw requests and responses it generates without reading the entirety of the source code.

Maybe you are generating HTTP requests through an API and need to see which headers it sets by default, or which headers are or are not getting set due to a misconfiguration or bug.

There are probably loads more use cases, and if I actually did programming for a living, I could probably list a lot more.

Post reply on HN