so will this detect residential proxies? how is that being done, I am getting hammered and its all legitimate normal ISP traffic.
To answer your first question, in my tests its around 50% of requests making it through.
Show HN: Aroma: Every TCP Proxy Is Detectable with RTT Fingerprinting
11–20 of 57 posts
Re: Show HN: Aroma: Every TCP Proxy Is Detectable with RTT Fingerprinting
#12I like this. I could see this being extra useful for people not using CDN's if they could easily plug it into nginx, haproxy and such. Currently for proxies I look for the proxy headers and also use a list of known proxy IP's but that is obviously nowhere near as complete as what you built. It might also be interesting to test assorted configurations of SSH forwards and MitM TLS caching proxies such as Squid SSL Bump.
Re: Show HN: Aroma: Every TCP Proxy Is Detectable with RTT Fingerprinting
#13This feels like something that’s a neat claim and will work against simple setups, but less accurate for more complicated scenarios (eg Tor). Then you’re really just relying on how accurate your knowledge of the proxies are.
Also, the readme has slightly incorrect logic I think:
> According to Special Relativity, information cannot travel faster than the speed of light. Therefore, if the round trip time (RTT) is 4ms, it's physically impossible for them to be farther than 2 light milliseconds away, which is approximately 600 kilometers.
It calls out the 33% for fiber but ignores that there’s not a straightline path between two points on the network and there could be wireless, cable, and DSL links somewhere on that hop.
Also, the controlled variable here is latency, not distance. Thus you can always increase latency through buffering and therefor you could be made to appear further than you are. And that buffering need not even be intentional - your perceived distance estimate will vary based upon queuing delays in intermediary depending on time of day (itself a fingerprint if you incorporate time-aware measurements, but a source of error if you don’t).
Fingerprinting is hard and I dislike the framing that it’s absolutely impossible to mask or that there’s not false positive and false negative error rates with the fingerprint.
Re: Show HN: Aroma: Every TCP Proxy Is Detectable with RTT Fingerprinting
#14I like this. I could see this being extra useful for people not using CDN's if they could easily plug it into nginx, haproxy and such. Currently for proxies I look for the proxy headers and also use a list of known proxy IP's but that is obviously nowhere near as complete as what you built. It might also be interesting to test assorted configurations of SSH forwards and MitM TLS caching proxies such as Squid SSL Bump…
I guess for this to work best you'd build your own CDN and have as many servers as possible. I have always dreamed of an Open Source CDN managed by a nonprofit and dedicated to offering CDN services for free or for a reasonable cost.
If you did the timings by comparing to other protocols, like TLS or HTTP you could do this with a single server, but that's a bit more complex than doing it on the same protocol since you have to account for more stuff, but it could be done, at the end of the day, my idea with Aroma was mostly to prove that it's possible, thanks for the feedback btw!
Re: Show HN: Aroma: Every TCP Proxy Is Detectable with RTT Fingerprinting
#15This feels like something that’s a neat claim and will work against simple setups, but less accurate for more complicated scenarios (eg Tor). Then you’re really just relying on how accurate your knowledge of the proxies are. Also, the readme has slightly incorrect logic I think: > According to Special Relativity, information cannot travel faster than the speed of light. Therefore, if the round trip time (RTT) is 4ms,…
About the straightline path I did think of that but apparently I forgot to address it when writing the README :p
The point I was trying to make is that if the RTT is low enough you can know the connection is being made from close, it's an upper bound, and making some assumptions you can get it lower, so it's not a way of knowing the exact distance but rather the max distance the connection can be made from. If someone is in Spain but they can't be more than 400km from Australia, something went terribly wrong somewhere hehe
In hindsight I think the issue with my explanation is that I was trying to explain the differences when fingerprinting two different protocols, but ended up going for a TCP-only approach since Fastly wouldn't expose to me the data I needed for the TLS and HTTP RTT. But in theory fingerprinting with protocol RTT difference where one protocol is proxied and the other is impossible to bypass, but this is only the theory.
I think I will edit the README in the future since I don't like how it turned out too much. Thanks for the feedback!
By the way, it detects Tor, I tested it ;D
Re: Show HN: Aroma: Every TCP Proxy Is Detectable with RTT Fingerprinting
#16Very clever, I like it.
When deployed on a popular server, one bit of "IP intelligence" this detector itself can gather is keep database of lowest-seen RTT per given source IP, maybe with some filtering - to cut out "faster-than-light" datapoints, gracefully update when actual network topology changes, etc.
That would establish a baseline, and from there, additional end-to-end RTT should become much more visible.
Re: Show HN: Aroma: Every TCP Proxy Is Detectable with RTT Fingerprinting
#17so will this detect residential proxies? how is that being done, I am getting hammered and its all legitimate normal ISP traffic.
To answer your first question, in my tests its around 50% of requests making it through.
Are you using a proxy? If you aren't that would be concerning, since false positives are way worse than false negatives.
If you are then it means the score is sometimes a bit lower and sometimes a bit higher than 0.1, which is the threshold for getting blocked.
If you want to know the exact score, you can check https://aroma.global.ssl.fastly.net/score
It's set at a low threshold since I want to avoid blocking regular users at all costs, I think the detection can be improved a lot by using more data and not a single division to calculate the score, in this case it's a somewhat simple PoC.
Thanks for taking the time to test it, I really appreciate it!
Re: Show HN: Aroma: Every TCP Proxy Is Detectable with RTT Fingerprinting
#18Very clever, I like it. When deployed on a popular server, one bit of "IP intelligence" this detector itself can gather is keep database of lowest-seen RTT per given source IP, maybe with some filtering - to cut out "faster-than-light" datapoints, gracefully update when actual network topology changes, etc. That would establish a baseline, and from there, additional end-to-end RTT should become much more visible.
First of all, thanks!
I imagine any big CDN implementing something like this could keep a database of all of this, combined with the old kind of IP intelligence and collecting not only RTT on other protocols like TLS, HTTP, IP (aka ping, and traceroutes too), TCP fingerprint, TLS fingerprint, HTTP fingerprint...
And with algorithms that combine and compare all these data points, I think very accurate models of the proxy could be made. And for things like credit card fraud this could be quite useful.
Re: Show HN: Aroma: Every TCP Proxy Is Detectable with RTT Fingerprinting
#19Earlier quoted context omitted.
To answer your first question, in my tests its around 50% of requests making it through.
Are you using a proxy? If you aren't that would be concerning, since false positives are way worse than false negatives. If you are then it means the score is sometimes a bit lower and sometimes a bit higher than 0.1, which is the threshold for getting blocked. If you want to know the exact score, you can check https://aroma.global.ssl.fastly.net/score It's set at a low threshold since I want to avoid blocking regula…
I'm testing using our residential proxies.
It's a super cool tool, I've been wondering about an open source tool doing this since reading about the technique in one of Nikolai Tschacher's blog posts years ago (https://incolumitas.com/pages/about/).
There's a few ways to work around this, but I think it's one of the best signals available to detect low-effort/common proxy providers.
Re: Show HN: Aroma: Every TCP Proxy Is Detectable with RTT Fingerprinting
#20Why would one want this? Are there particular situation(s) that it's desirable to detect a TCP proxy? Does presence of a TCP proxy indicate some adverserial behaviour? E.g. surveillance, censorship, a particular attack?