Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: HTTP Mock – Intercept, debug and mock HTTP(S) with zero setup

httptoolkit.tech

Post reply on HN
Print view View original

31–40 of 86 posts

Re: Show HN: HTTP Mock – Intercept, debug and mock HTTP(S) with zero setup

#31

Post by pimterry »

dessant wrote: post #29
pimterry wrote: post #27

Earlier quoted context omitted.

Yes, definitely! You can follow progress by subscribing to this GH issue: https://github.com/httptoolkit/feedback/issues/10 (or signing up to the general HTTP Toolkit mailing list ofc). In the meantime, it's quite possible to intercept Android traffic, it's just that you'll have to do the proxy & certificate setup yourself.

That's great, thanks for the link! In recent Android versions it has become difficult to inspect the traffic precisely because certificates installed by users are no longer trusted, so one has to either root the phone or modify the apk. Do you plan to automate app repackaging with the needed changes, or is there a better method for apps to trust local certificates?

> Do you plan to automate app repackaging with the needed changes, or is there a better method for apps to trust local certificates?

Eventually I'm aiming for the former, I'm fairly confident there is no better method. To start with, it'll be an Android app that configures your proxy settings (acting a VPN) and walks you through adding the certificate to the user store, and for your own apps I believe that's useful already: you just need to enable user stores in your app config (https://developer.android.com/training/articles/security-con...), and everything will Just Work.

For other apps though, on new-ish Android apps it's more difficult, as you say. My plan is to try and rewrite them, _probably_ as a Pro feature, but TBC. It should be a matter of:

- Get hold of the app APK (I think you can pull it with some adb tricks, slightly unclear)

- Edit the XML config to enable the user cert store, change the app id too (so it doesn't replace or have signature conflicts with the real app)

- Repackage & reinstall the resulting APK over adb

I haven't tested this yet though, so who knows!

Re: Show HN: HTTP Mock – Intercept, debug and mock HTTP(S) with zero setup

#32

Post by pimterry »

mcx wrote: post #30

Would be cool to have something like this for debugging outgoing container traffic in Kubernetes.

Docker is on the list: https://github.com/httptoolkit/feedback/issues/1 (you can subscribe to that for updates). Would that work for your setup?

Re: Show HN: HTTP Mock – Intercept, debug and mock HTTP(S) with zero setup

#33

Post by debatem1 »

pimterry wrote: post #22
meritt wrote: post #19

How is this achieving https interception with zero setup? Does that not require a system-level root CA to be installed and trusted?

> Does that not require a system-level root CA to be installed and trusted? No, happily :-) The trick is that it starts the application to be intercepted for you, so it can control it a little. It then does some magic to get that specific instance of the application to trust the certificate. There's a lot going on there, but as an example: Chrome has a --ignore-certificate-errors-spki-list to inject the hashes of ext…

So, I'm confused. Does this work on a set of programs where you know eg the right command line parameters to give them or on all programs?

For example, let's say I have a program which uses the openssl API to set a custom trust store. Do you mess with openssl to make sure your cert is in there, or will this break?

Re: Show HN: HTTP Mock – Intercept, debug and mock HTTP(S) with zero setup

#34

Post by debatem1 »

pimterry wrote: post #17
nachi wrote: post #14

Earlier quoted context omitted.

What about intercepting requests from other devices on the same network? Fiddler and Charles both let you set a proxy on Android/iOS devices pointing to your instance.

HTTP Toolkit can definitely intercept Android & iOS or other network devices, it just can't do it for you automatically (yet. Watch this space: https://github.com/httptoolkit/feedback/issues/10 ). If you want to do intercept something manually, there's general instructions under the 'Anything' interception option. In short: - Set the proxy on the device to point at your machine's local IP, usually on port 8000 (activ…

I know it isn't exactly what you're doing, but you might want to look at nogotofail on the Android side. It has all the per-app goodness baked in, and you could probably borrow its approach.

Re: Show HN: HTTP Mock – Intercept, debug and mock HTTP(S) with zero setup

#35

Post by pimterry »

debatem1 wrote: post #33
pimterry wrote: post #22

Earlier quoted context omitted.

> Does that not require a system-level root CA to be installed and trusted? No, happily :-) The trick is that it starts the application to be intercepted for you, so it can control it a little. It then does some magic to get that specific instance of the application to trust the certificate. There's a lot going on there, but as an example: Chrome has a --ignore-certificate-errors-spki-list to inject the hashes of ext…

So, I'm confused. Does this work on a set of programs where you know eg the right command line parameters to give them or on all programs? For example, let's say I have a program which uses the openssl API to set a custom trust store. Do you mess with openssl to make sure your cert is in there, or will this break?

It works on a set of programs, plus terminal interception that works on anything (more or less) that you spawn from the given terminal.

For that terminal interception, HTTP Toolkit injects a lot of environment variables, including adding some wrappers to your path, which allow it to inject into a _lot_ of places. It sets `SSL_CERT_FILE` for example, which reconfigures the default CA for any processes started from that terminal that use OpenSSL. Again, all scoped to just this one terminal window.

In the case where you specifically manage the trust store as you describe I suspect that'll break (the requests will fail, but they'll still appear in HTTP Toolkit as just 'Certificate rejected for ABC.com', without the full details). The vast majority of applications don't do that though; they rely on the environment to tell them who to trust, and HTTP Toolkit creates the environment. For specific cases that don't, I also include a bunch of extra overrides and injections, so if there's anything that comes up that doesn't fit it's fairly easy to work around it.

It's all open-source, so you can look through the specifics if you like. The core terminal interception setup logic is https://github.com/httptoolkit/httptoolkit-server/blob/maste.... That spawns a terminal and injects the environment variables from https://github.com/httptoolkit/httptoolkit-server/blob/maste..., which notably add a bunch of things from https://github.com/httptoolkit/httptoolkit-server/tree/maste... to your PATH/PYTHONPATH/RUBYLIB/etc.

Re: Show HN: HTTP Mock – Intercept, debug and mock HTTP(S) with zero setup

#36

Post by pimterry »

debatem1 wrote: post #34
pimterry wrote: post #17

Earlier quoted context omitted.

HTTP Toolkit can definitely intercept Android & iOS or other network devices, it just can't do it for you automatically (yet. Watch this space: https://github.com/httptoolkit/feedback/issues/10 ). If you want to do intercept something manually, there's general instructions under the 'Anything' interception option. In short: - Set the proxy on the device to point at your machine's local IP, usually on port 8000 (activ…

I know it isn't exactly what you're doing, but you might want to look at nogotofail on the Android side. It has all the per-app goodness baked in, and you could probably borrow its approach.

Interesting! Yeah that definitely looks relevant, I'll take a look.

Re: Show HN: HTTP Mock – Intercept, debug and mock HTTP(S) with zero setup

#37

Post by chrisweekly »

ORioN63 wrote: post #28
Nodraak wrote: post #10

Earlier quoted context omitted.

Would not be able to exactly explain the features and differences, but https://mitmproxy.org/ is a similar project you might be interested in.

mitmproxy is indeed a similar project. It offers a dashboard similar to devtools (no modifying in there though). You can, however, write small python scripts that have read-write access to the http flow (request and response objects). I guess it's a more low level tool, because it doesn't provide mocking and other helpful tools out of the box.

mitmproxy (/mitmdump) is awesome and powerful, and is ideal for integrating w/ other cli tools (like lnav) in chained scripts written in your language of choice. highly recommended.

Re: Show HN: HTTP Mock – Intercept, debug and mock HTTP(S) with zero setup

#38

Post by hartator »

gmanley wrote: post #24
hartator wrote: post #20

Love it. We definitely need a fiddler in macosx.

Charles is pretty good, though it’s paid. It’s definitely nice to have an opensource tool, though.

Charles is okay. But when you have been used to Fiddler is hard to go back.

Re: Show HN: HTTP Mock – Intercept, debug and mock HTTP(S) with zero setup

#39

Post by xyzzy_plugh »

pimterry wrote: post #11
mariocesar wrote: post #4

There are full opensource alternative for this? I like the product, I'm testing it right now. But I'm curious if there is a way to accomplish something like this with existing opensource tools or a similar full opensource project.

HTTP Toolkit is fully open source, primarily AGPL: https://github.com/httptoolkit/ . That includes the Pro version too - it's not open core, it's just open source all the way down. There are other tools that exist anyway, but all the big ones (Fiddler & Charles, for Windows & OSX respectively) are very much closed source. The other similar open source competitor I'm familiar with is James Proxy ( https://github.com/j…

The parent was likely referring to AGPL, a somewhat-encumbered copyleft license, which most folks working in proprietary software are hesitant to touch. They are probably looking for a non-copyleft license.

Though, as a development tool, I don't see any issues using AGPL software.

Re: Show HN: HTTP Mock – Intercept, debug and mock HTTP(S) with zero setup

#40

Post by pault »

pimterry wrote: post #2

Hey HN! I've been building this for a while, but up until now it's been a read-only HTTP debugging tool, and today's my first foray into proper mocking, which opens up a whole new world of fun (and practical use cases). Let me know what you think :-)

I hope this works out! Using fiddler us such a frustrating experience. The documentation is practically non existent and the ux is atrocious, and as far as I can tell the closest thing to a manual is a paid ebook by the developer (I was so pissed when I discovered this). I would be very happy to have an open source alternative!
Post reply on HN