Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Aroma: Every TCP Proxy Is Detectable with RTT Fingerprinting

github.com

Post reply on HN
Print view View original

51–57 of 57 posts

Re: Show HN: Aroma: Every TCP Proxy Is Detectable with RTT Fingerprinting

#51

Post by dlenski »

JDye wrote: post #47
dlenski wrote: post #46

Earlier quoted context omitted.

Came here to ask the same thing. Why do I _care_ if connections to my server come from a TCP proxy? Particularly when a VPN is _not_ observable in a similar way? Is there some class of bad actors who extensively use TCP proxies and not only _don't_ use VPNs, but would incur large costs in switching to them?

Web scrapers maybe aren't "bad actors", but many sites dont want them. They'll use tons of TCP proxies which route them through a rotating pool of end user devices (mobiles, routers, etc...). Its not really possible to block these IPs as you'd also be blocking legitimate customers so other ways to detect and block are required.

Can't/won't these scrapers just switch to using VPNs or sshuttle or basically anything else that doesn't leak timing info about termination of TCP vs HTTP?

Re: Show HN: Aroma: Every TCP Proxy Is Detectable with RTT Fingerprinting

#52

Post by JDye »

dlenski wrote: post #51
JDye wrote: post #47

Earlier quoted context omitted.

Web scrapers maybe aren't "bad actors", but many sites dont want them. They'll use tons of TCP proxies which route them through a rotating pool of end user devices (mobiles, routers, etc...). Its not really possible to block these IPs as you'd also be blocking legitimate customers so other ways to detect and block are required.

Can't/won't these scrapers just switch to using VPNs or sshuttle or basically anything else that doesn't leak timing info about termination of TCP vs HTTP?

Not really. You can have 100,000 IPs from proxies or use VPNs and have only 5 egress IPs.

Anybody who wants to stop the scraper could get browser fingerprints, cross reference similar ones with those IPs and quite safely ban them as its highly likely theyre not a legitimate customer.

Its a lot harder to do it for the 100k IPs because those IPs will also have legitimate customer traffic on them and its a lot more likely the browser fingerprint could just be legitimate.

The risk of false postives (blocking real people) is usually higher than just allowing the scrapers and the incetives of a lot of sites arent aligned with stopping scrapers anyway. Think eccommerce, do they _really_ care if the product is being sold to scalpers or real customers? If anything, that behaviour can raise perception of their brand, increase demand, increase prices.

This tool should have less false positives than most, so maybe it will see more adoption than others (TCP fingerprinting for example) but I dont think this is going to affect anyone doing scraping seriously/at scale.

Re: Show HN: Aroma: Every TCP Proxy Is Detectable with RTT Fingerprinting

#53

Post by dlenski »

JDye wrote: post #52
dlenski wrote: post #51

Earlier quoted context omitted.

Can't/won't these scrapers just switch to using VPNs or sshuttle or basically anything else that doesn't leak timing info about termination of TCP vs HTTP?

Not really. You can have 100,000 IPs from proxies or use VPNs and have only 5 egress IPs. Anybody who wants to stop the scraper could get browser fingerprints, cross reference similar ones with those IPs and quite safely ban them as its highly likely theyre not a legitimate customer. Its a lot harder to do it for the 100k IPs because those IPs will also have legitimate customer traffic on them and its a lot more like…

> Not really. You can have 100,000 IPs from proxies or use VPNs and have only 5 egress IPs.

Why…?

If I can run a proxy exit node on 100k residential IPs, why can't I run a VPN server on 100k residential IPs?

There is no additional technical complexity or resource consumption from the VPN server compared to the proxy server.

Re: Show HN: Aroma: Every TCP Proxy Is Detectable with RTT Fingerprinting

#54

Post by benmmurphy »

dlenski wrote: post #48
Rasbora wrote: post #22

This is the core concept of how proxies are detected via services like https://layer3intel.com/tripwire or https://spur.us/monocle/ The difference in min TCP RTT and min RTT to respond to a websocket payload is a dead giveaway that there's a middlebox terminating TCP somewhere along the path. You can bypass this by sourcing your request within 30ms of wherever TCP is being terminated, anything under that threshold co…

> I expect it won't be long until someone deploys the first proxy service that handles the initial CONNECT payload in the kernel before offloading packet forwarding to an eBPF script that will proxy packets between hosts at layer 3, making this fingerprinting technique obsolete. https://github.com/sshuttle/sshuttle basically works like this. I've used it for many years. I don't think it'll be possible to detect using…

sshuttle as described sounds like a normal CONNECT proxy which this is able to detect: https://sshuttle.readthedocs.io/en/stable/how-it-works.html

like its similar to connect or socks proxy except it is using SSH as a transport layer instead of TCP as a transport layer and its doing it transparently without having applications to be written to use the proxy. but if you are just converting TCP packets into a datastream and then sending them somewhere else where you convert them back to TCP packets then this is what this TCP RTT strategy is fundamentally meant to detect. i suspect the TCP only RTT thing works because of the delayed ack behaviour of most operating systems and this will still happen with sshuttle unless you are explicitly using quick-ack. also, quick-ack just works around the TCP-RTT issue and not the differences in timing between TCP and TLS or other higher protocols. i think if you are testing for other RTT differences then quick-ack would make them more obvious.

on the server side sshuttle just uses normal tcp sockets and nothing magic (https://github.com/sshuttle/sshuttle/blob/master/sshuttle/ss...)

also, if you have an sshuttle proxy this site cannot detect it may be due to how close the server is to the client. i have a CONNECT based proxy it is able to detect around 5% of the time (maybe only that often due to a bug) but this is because there is probably less than 10ms latency between the proxy and the client and probably around 50ms latency between the proxy and the server for some reason (?).

Re: Show HN: Aroma: Every TCP Proxy Is Detectable with RTT Fingerprinting

#55

Post by benmmurphy »

dlenski wrote: post #53
JDye wrote: post #52

Earlier quoted context omitted.

Not really. You can have 100,000 IPs from proxies or use VPNs and have only 5 egress IPs. Anybody who wants to stop the scraper could get browser fingerprints, cross reference similar ones with those IPs and quite safely ban them as its highly likely theyre not a legitimate customer. Its a lot harder to do it for the 100k IPs because those IPs will also have legitimate customer traffic on them and its a lot more like…

> Not really. You can have 100,000 IPs from proxies or use VPNs and have only 5 egress IPs. Why…? If I can run a proxy exit node on 100k residential IPs, why can't I run a VPN server on 100k residential IPs? There is no additional technical complexity or resource consumption from the VPN server compared to the proxy server.

for phones its a bit difficult because i don't think you can egress out ip traffic without root or jailbreak on iphone and iOS. but i guess on desktop this should be possible

Re: Show HN: Aroma: Every TCP Proxy Is Detectable with RTT Fingerprinting

#56

Post by JDye »

dlenski wrote: post #53
JDye wrote: post #52

Earlier quoted context omitted.

Not really. You can have 100,000 IPs from proxies or use VPNs and have only 5 egress IPs. Anybody who wants to stop the scraper could get browser fingerprints, cross reference similar ones with those IPs and quite safely ban them as its highly likely theyre not a legitimate customer. Its a lot harder to do it for the 100k IPs because those IPs will also have legitimate customer traffic on them and its a lot more like…

> Not really. You can have 100,000 IPs from proxies or use VPNs and have only 5 egress IPs. Why…? If I can run a proxy exit node on 100k residential IPs, why can't I run a VPN server on 100k residential IPs? There is no additional technical complexity or resource consumption from the VPN server compared to the proxy server.

I don't mean that you can't do it, just that there is no company offering it so right now those are the only two options.

It's something we're experimenting with currently. the other commenter is right about apple products, but on android, desktop, etc... it's pretty easy.

Post reply on HN