Live data from Hacker News

Ask HN: How did the internet discover my subdomain?

news.ycombinator.com

31–40 of 322 posts

Re: Ask HN: How did the internet discover my subdomain?

#31

Certificate Transparency logs, or they don't actually know the domain name: just port-scanning[1] then making requests to open web ports. [1] Turns out you can port-scan the entire internet in under 5 minutes: https://github.com/robertdavidgraham/masscan

Port scanning usually can't discover subdomains. Most servers don't expose the of the domains they server content for. In case of HTTP they usually only serve the subdomain content if the Host: request-header includes it.

Re: Ask HN: How did the internet discover my subdomain?

#35
post #31

Certificate Transparency logs, or they don't actually know the domain name: just port-scanning[1] then making requests to open web ports. [1] Turns out you can port-scan the entire internet in under 5 minutes: https://github.com/robertdavidgraham/masscan

Port scanning usually can't discover subdomains. Most servers don't expose the of the domains they server content for. In case of HTTP they usually only serve the subdomain content if the Host: request-header includes it.

Most servers just listen on :80 and respond to all requests. Almost nobody checks the host header intentionally, it's just a happy mistake if they use a reverse proxy.

You can often decloak servers behind Cloudflare because of this.

But OP's post already answered their question: someone scanned ipv4 space. And what they mean is that a server they point to via DNS is receiving requests, but DNS is a red herring.

Re: Ask HN: How did the internet discover my subdomain?

#36
post #31

Certificate Transparency logs, or they don't actually know the domain name: just port-scanning[1] then making requests to open web ports. [1] Turns out you can port-scan the entire internet in under 5 minutes: https://github.com/robertdavidgraham/masscan

Port scanning usually can't discover subdomains. Most servers don't expose the of the domains they server content for. In case of HTTP they usually only serve the subdomain content if the Host: request-header includes it.

I could be wrong, but the Palo Alto scanner says it's using global ipv4 space, so not using DNS at all. So actually the subdomain has not been discovered at all.

Re: Ask HN: How did the internet discover my subdomain?

#37
If a HTTPS service should be hard to discover, an easy way is to hide it behind a subdirectory. Something like https://subdomain.domain.example/hard_to_find_secret_string.

Another option are wildcard certificates.

This obviously can't be the only protection. But if an attacker doesn't know about a service, or misses it during discovery, they can't attack it.

Re: Ask HN: How did the internet discover my subdomain?

#38
post #31

Certificate Transparency logs, or they don't actually know the domain name: just port-scanning[1] then making requests to open web ports. [1] Turns out you can port-scan the entire internet in under 5 minutes: https://github.com/robertdavidgraham/masscan

Port scanning usually can't discover subdomains. Most servers don't expose the of the domains they server content for. In case of HTTP they usually only serve the subdomain content if the Host: request-header includes it.

How deep in the domain hierarchy you are doesn't matter from a network layer: a bare tld (yes this exists), a normal domain, a subdomain, a sub-subdomain, etc can all be assigned different IPs and go different places. You can issue a GET against / for any IP you want (like we see in the logs OP posted). The only time this would actually matter is if a host at an address is serving content for multiple hostnames and depends on the Host header to figure out which one to serve -- but even those will almost always have a default.

Re: Ask HN: How did the internet discover my subdomain?

#39
post #24

LPT, this is an object lesson in the weakness of security through obscurity

I mean you could argue that this is more of a multi-factor authentication lesson.

Just knowing 1 "secret"— a subdomain in this case —shouldn't get you somewhere you shouldn't.

In general you should always assume that any password has been (or could be) compromised. So in this case, more factors should be involved such as IP restricting for access, an additional login page, certificate validation, something...

Re: Ask HN: How did the internet discover my subdomain?

#40
post #31

Earlier quoted context omitted.

Port scanning usually can't discover subdomains. Most servers don't expose the of the domains they server content for. In case of HTTP they usually only serve the subdomain content if the Host: request-header includes it.

Most servers just listen on :80 and respond to all requests. Almost nobody checks the host header intentionally, it's just a happy mistake if they use a reverse proxy. You can often decloak servers behind Cloudflare because of this. But OP's post already answered their question: someone scanned ipv4 space. And what they mean is that a server they point to via DNS is receiving requests, but DNS is a red herring.

This really depends on the setup. Most web servers host multiple virtual hosts. IP addresses are expensive.

If you're deploying a service behind a reverse proxy, it either must be only accessible from the reverse proxy via an internal network, or check the IP address of the reverse proxy. It absolutely must not trust X-Forwarded-For: headers from random IPs.

Post reply on HN