I've been looking to use mitmproxy to capture traffic in a project, but the serialization format is something called TNetStrings [1], which postfixes types and is thus not streamable. There seem to be some efforts to switch flow captures to use SQLite [2] or Protobuf [3], although PCAP would also be nice, I think.
Mitmproxy 8
21–30 of 63 posts
Re: Mitmproxy 8
#22Earlier quoted context omitted.
You can use this to debug stuff happening over TLS connections, but there are also other practical ways to use it. For example, install the CA certificate on your phone (with root if you have Android) and set up a WiFi network that transparently proxies everything through mitmproxy. Let it collect data for a while and you'll be surprised how many random servers even well known apps try to contact. You'll also be able…
Since Android Nougat I think you cannot install user certificates unless you: 1. Root your device. 2. Add the certificate to the APK manifest. I'm not well versed in Android changes, this is just the conclusion I came to when Nougat came out and haven't tried it again since. If there's an easier way to use MITM with Android without modifying the APK / rooting your device, I'd love to know!
This is true, by default Android apps do not trust user-installed certificate authorities. IMO the easiest solution if you're doing security testing on a dedicated device is MagiskTrustUserCerts[1]. If you're not testing on a dedicated device or you don't want to root the device, I'd recommend using the objection[2] tool which has a guided mode for patching an apk, and you can modify the manifest to add your CA or to trust all user-installed CAs.
[1]: https://github.com/NVISOsecurity/MagiskTrustUserCerts
[2]: https://github.com/sensepost/objection/wiki/Patching-Android...
Re: Mitmproxy 8
#23I've been looking to use mitmproxy to capture traffic in a project, but the serialization format is something called TNetStrings [1], which postfixes types and is thus not streamable. There seem to be some efforts to switch flow captures to use SQLite [2] or Protobuf [3], although PCAP would also be nice, I think. [1] https://tnetstrings.info/ [2] https://github.com/mitmproxy/mitmproxy/issues/3075 [3] https://github.…
There’s an addon that supports PCAP files [0]. It works quite well in my experience.
Re: Mitmproxy 8
#24This is cool. I’m learning a lot reading through the docs. Why is subversion so instructive? I’ve read about SSL certificates a gazillion times and it’s simple enough in theory but for some reason this is making everything click. What’s the use most common use case for something like this, by the way?
You can use this to debug stuff happening over TLS connections, but there are also other practical ways to use it. For example, install the CA certificate on your phone (with root if you have Android) and set up a WiFi network that transparently proxies everything through mitmproxy. Let it collect data for a while and you'll be surprised how many random servers even well known apps try to contact. You'll also be able…
I haven't tried it for replacing TLS certificates, but you do not need a rooted phone to mess with 3rd party apps on android. You can unpack an APK and re-sign it using your own key, allowing you to make whatever modifications to the APK you want. In addition to making whatever changes you want to directly (such as just replacing the pinned certificates), you can update the manifest to set the android:debuggable flag to true, which allows you to access and modify the apps private files through ADB.
The main downside of this approach is you cannot do an in-place replacement of already installed app, as your version is signed by a different key. You need to uninstall the previous version first, causing you to lose existing app data. At one point, I was able to work around this using Android's backup feature, but I never bothered to retain data since then since it was too much of a hassle and turned out not to matter that much for any of the apps I was playing with.
Re: Mitmproxy 8
#25Earlier quoted context omitted.
You can use this to debug stuff happening over TLS connections, but there are also other practical ways to use it. For example, install the CA certificate on your phone (with root if you have Android) and set up a WiFi network that transparently proxies everything through mitmproxy. Let it collect data for a while and you'll be surprised how many random servers even well known apps try to contact. You'll also be able…
Since Android Nougat I think you cannot install user certificates unless you: 1. Root your device. 2. Add the certificate to the APK manifest. I'm not well versed in Android changes, this is just the conclusion I came to when Nougat came out and haven't tried it again since. If there's an easier way to use MITM with Android without modifying the APK / rooting your device, I'd love to know!
You can install certificates just fine, but apps need to opt in to use user certificates. You can MitM apps that have this flag enabled, but very few mobile developers seem to even know what TLS is, let alone how and why you should change trust store settings.
With root access, you can put your certificate in the root store, which most apps use for validation. I wrote a blog about it around the time Android 7 came out. Some apps (either not enough or too many, depending on your perspective) leverage Android's certificate pinning feature to connect a domain to a specific TLS certificate. You can't MitM those apps without either modifying the APK or hijacking them via Frida.
Lastly, if you want to MitM Firefox, you need to enable their secret debug menu (available on some builds) to enable the flag that enables user certificates. Flagging the relevant setting in about:config does not work and about:config isn't even accessible on stable builds of Firefox for Android.
Sadly, there's no good way to MitM an Android device without full system access. Android has always been quite annoying to use custom CA certs for, but since Android 7 it's pretty much impossible without root access. I'm not entirely sure why they went this route instead of sticking to their existing warnings (i.e. a notification every time a custom CA gets used, like in Android 4, and a constant "your network may be monitored" label in the notification tray like in Android 5+).
Re: Mitmproxy 8
#26Is there any benchmark against squid proxy in regards of cpu and memory usage? Anyone has expiernce with it or can recommend something else?
Squid is meant to be used as a general proxy for things like caching to help your local network save bandwidth, etc. Mitmproxy is more of a pen testing/security tool. They’re very different things and not something you would be comparing benchmarks to try to choose one.
Re: Mitmproxy 8
#27I've used several other apps like Mitmproxy, and some of them have nicer UIs... but Mitmproxy is the only one I've found that supports MITM'ng connections using client certificates.
Re: Mitmproxy 8
#28This is cool. I’m learning a lot reading through the docs. Why is subversion so instructive? I’ve read about SSL certificates a gazillion times and it’s simple enough in theory but for some reason this is making everything click. What’s the use most common use case for something like this, by the way?
Not sure about the most common, but I used it to inspect some requests that an app was making so I could find the needed parameters to get the data from the App’s backend servers in a tool I was developing. I was able to proxy the app’s connection and inspect the traffic as I used the app and was easily able to see what it was doing and replicated the parts I needed.
Re: Mitmproxy 8
#29Earlier quoted context omitted.
Since Android Nougat I think you cannot install user certificates unless you: 1. Root your device. 2. Add the certificate to the APK manifest. I'm not well versed in Android changes, this is just the conclusion I came to when Nougat came out and haven't tried it again since. If there's an easier way to use MITM with Android without modifying the APK / rooting your device, I'd love to know!
You can install certificates just fine, but apps need to opt in to use user certificates. You can MitM apps that have this flag enabled, but very few mobile developers seem to even know what TLS is, let alone how and why you should change trust store settings. With root access, you can put your certificate in the root store, which most apps use for validation. I wrote a blog about it around the time Android 7 came ou…
You are right, I didn't phrase it correctly, what I is meant that since Nougat I'm not aware of way to force system-wide use of a user certificate without gaining root access.
Thanks for the Firefox tip as well, it's been a while since I used Mitmproxy and I see things have not become easier since.
Re: Mitmproxy 8
#30Earlier quoted context omitted.
Since Android Nougat I think you cannot install user certificates unless you: 1. Root your device. 2. Add the certificate to the APK manifest. I'm not well versed in Android changes, this is just the conclusion I came to when Nougat came out and haven't tried it again since. If there's an easier way to use MITM with Android without modifying the APK / rooting your device, I'd love to know!
This is true, by default Android apps do not trust user-installed certificate authorities. IMO the easiest solution if you're doing security testing on a dedicated device is MagiskTrustUserCerts[1]. If you're not testing on a dedicated device or you don't want to root the device, I'd recommend using the objection[2] tool which has a guided mode for patching an apk, and you can modify the manifest to add your CA or to…
Neat! Both Magisk (which I might use on an older phone which I don't mind rooting) and Objection are new to me, thank you!