I'm confused. Mitmproxy shows me request/response bodies and lets me edit and replay requests. Those seem like fundamental features (Fiddler, Burp, mitmproxy all seem to have them). I don't see how this is done with nginx reverse_proxying and logging, or is that coming in part 2 or 3 maybe?
Making your own web debugging proxy
11–20 of 24 posts
Re: Making your own web debugging proxy
#12You can do that using MiTM Proxy as well, as explained here: https://dadario.com.br/mitming-ssl-tls-connections/
I use mitmproxy and mitmdump a lot. I really recommend it. Someone else said it's not easy to hack on, I do not agree. It's a really approachable project and great if you're doing anything HTTP related. That said, there are some pain points:
* libmproxy is not recommended for external projects. Instead you should use "inline scripts" (embedding functionality in mitmproxy). This is a pain point for me since I wanted to work on the captured streams in an external program that already existed and I shouldn't need to run mitmdump to do that work. The dumped streams are not in a standard format either, they're serialized python objects and dumps from different mitmproxy versions sometimes break the format.
* Performance. Having more than a couple of concurrent requests at the time tends to eat CPU. Running the requests of one browser through it is fine. Running multiple browser instances (e.g., using the new headless library in chromium for automated testing) through it, continuously performing requests is not a good idea. Memory footprint is also quite high, but is not a limiting factor for me.
I would like it if the dump format was a standard format and if libmproxy was a stable API.
Re: Making your own web debugging proxy
#13You can do that using MiTM Proxy as well, as explained here: https://dadario.com.br/mitming-ssl-tls-connections/
I use mitmproxy and mitmdump a lot. I really recommend it. Someone else said it's not easy to hack on, I do not agree. It's a really approachable project and great if you're doing anything HTTP related. That said, there are some pain points: * libmproxy is not recommended for external projects. Instead you should use "inline scripts" (embedding functionality in mitmproxy). This is a pain point for me since I wanted t…
Performance is actually the reason I started this project.
A friend of mine was creating a product based on mitmproxy for a client and was running in to performance projects. He asked me for advice and I pointed him to just how little work it was to do exactly what he wanted with openresty, instead of mitmproxy.
If you want everything, mitmproxy might be for you. If you want fast, minimal, hackable, and not a pile of python then openresty and my approach might be for you.
Re: Making your own web debugging proxy
#14Earlier quoted context omitted.
I found easy to get started, it's as simple as the post. The real benefit I see is to plug arbitrary python code to hack requests and responses. It's very good, but I've noticed some crashes in my experiments too.
That's what the benefit of using openresty is. I can print and hack on http requests in an already mature ecosystem. It's two ways to do the same thing, but I like my way, and mine uses software that is production ready as a proxy :)
Open testy is awesome for lots of things. However, if I'm doing HTTP debugging, I use curl, Charles, ZAP ( https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Proje... ), tcpdump, and wireshark, in order from simplest to most complex problems.
Re: Making your own web debugging proxy
#15If you just want a quick proxy to inspect traffic, apache with mod_dumpio¹ always seemed the quickest and easiest way to do it: just proxypass your traffic and
LogLevel dumpio:trace7
DumpIOInput On
DumpIOOutput On
Re: Making your own web debugging proxy
#16Hey.
About HTTPS proxy, i can offer you a better way, rather than creating your own CA, generating certs for any domain which is too much of work & configuration + compiling OpenSSL.
I have done that already, as free service working on this address: https://ca.parasite.io
You can easily implement with LUA module to download certs for any domain & download it as Zip or JSON or pfx. Contains all files you need. root, intermediate and target cert with private keys of course. As the owner/developer, that domain and service is going to work for years at least till 2027 (my root cert's expiry date).
Note: Created certs has a 60 mins of cache (nginx) to improve performance. You don't want to download each certificate for all static files in a single request.
Re: Making your own web debugging proxy
#17Im all in favour of owning your stack completely, but sometimes you need to be pragmatic for the use-case.
I occasionally need to debug traffic. Not often, but occasionally.
For me, the ~$5 on Cellist (http://cellist.patr0n.us/index.html) was a no-brainer.
Re: Making your own web debugging proxy
#18Hey. About HTTPS proxy, i can offer you a better way, rather than creating your own CA, generating certs for any domain which is too much of work & configuration + compiling OpenSSL. I have done that already, as free service working on this address: https://ca.parasite.io You can easily implement with LUA module to download certs for any domain & download it as Zip or JSON or pfx. Contains all files you need. root, i…
Is this what it looks like? A service asking people to download and install a new root CA certificate?
Don't ever do that.
Re: Making your own web debugging proxy
#19Im all in favour of owning your stack completely, but sometimes you need to be pragmatic for the use-case. I occasionally need to debug traffic. Not often, but occasionally. For me, the ~$5 on Cellist ( http://cellist.patr0n.us/index.html ) was a no-brainer.
There's also mitmproxy [0] Charles (which apparently was the inspiration for this post) [1] and Fiddler [2].
Re: Making your own web debugging proxy
#20Hey. About HTTPS proxy, i can offer you a better way, rather than creating your own CA, generating certs for any domain which is too much of work & configuration + compiling OpenSSL. I have done that already, as free service working on this address: https://ca.parasite.io You can easily implement with LUA module to download certs for any domain & download it as Zip or JSON or pfx. Contains all files you need. root, i…
Is this what it looks like? A service asking people to download and install a new root CA certificate ? Don't ever do that.
Fiddler does this, but locally - I should check how long the expiry is, though.