There's also PCAPdroid [1] which you can run straight from your phone with no root. Works with https traffic too when you enable the mitm setting.
Scraping data from an app: real world example
21–26 of 26 posts
Re: Scraping data from an app: real world example
#22Earlier quoted context omitted.
Why isn't there a site-controlled fallback setting for this? Does this not make sense? Abu given website's beet interest is to continue to be reachable.
Every escape hatch in the certificate validation is also an additional avenue for attack. For example, using a DNS record to override certificate pins makes DNS cache poisoning much more valuable to the attacker.
Got it, thanks @tremon.
Re: Scraping data from an app: real world example
#23Earlier quoted context omitted.
Can someone suggest some resources to understand the additional work needed to decrypt the pinned https traffic?
Android: https://httptoolkit.tech/blog/frida-certificate-pinning/ iOS: https://nabla-c0d3.github.io/blog/2013/08/20/intercepting-th...
This is really helpful, thanks!
Re: Scraping data from an app: real world example
#24Most app have some form of SSL pinning system in place which means that you have to perform additional work to allow the proxy to decrypt the HTTPS traffic.
"Most app have some form of SSL pinning system in place..." I would like to see the data, if any, supporting this statement. I would expect some apps would use pinning, but most would not. Google recommends against it. https://developer.android.com/training/articles/security-ssl...
I reverse engineer Android apps for work and pinning is present in all but the lowest effort apps I encounter.
Re: Scraping data from an app: real world example
#25Earlier quoted context omitted.
Why isn't there a site-controlled fallback setting for this? Does this not make sense? Abu given website's beet interest is to continue to be reachable.
Every escape hatch in the certificate validation is also an additional avenue for attack. For example, using a DNS record to override certificate pins makes DNS cache poisoning much more valuable to the attacker.
Every layer of security is also an additional accessibility hurdle.
Re: Scraping data from an app: real world example
#26That is really nice. The last time I attempted scraping an app was using an android emulator (bluestacks), then using maybe Wireshark or Charles for getting the API endpoint. It didn't work for some reason though. I don't remember the exact error and I am kinda skeptical about app scraping being this easy.
I recently did something similar with good results (I found the api endpoints I was interested in) using the official Android emulator and https://github.com/mitmproxy/mitmproxy I did have to jump through some hoops with the emulator and pushing my own ssl cert to it's RO system partition. But it was a few commands and easy enough.
Thank you. I am going to try out your solution :) appreciate it.