Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How did the internet discover my subdomain?

news.ycombinator.com

Post reply on HN
Print view View original

121–130 of 322 posts

Re: Ask HN: How did the internet discover my subdomain?

#121

Post by melevittfl »

p0w3n3d wrote: post #91
paxys wrote: post #62

Not sure why everyone is going on about certificate transparency logs when the answer is right there in the user agent. The company is scanning the ipv4 space and came upon your IP and port.

Finding IP does not mean finding the domain. When doing HTTP request to IP you specify the domain you want to connect to. For example you can configure your /etc/hosts to have xxxnakedhamsters.google.com pointing to 8.8.8.8 and make the http request, which will cause Google getting the domain request (i.e. header Host: xxxnakedhamsters.google.com) and it will refuse it or try to redirect to http. Of course it's only…

But there's no evidence in the OP's post that they have, in fact, discovered the domain. The only thing posted is that there is a GET request to a listening web server.

The OP and all the people talking about certificates are making the same assumption. Namely that the scanning company discovered the DNS name for the server and tried to connect. When, if fact, they simply iterate through IP address blocks and make get requests to any listening web servers they find.

Re: Ask HN: How did the internet discover my subdomain?

#123

Post by ralferoo »

If you're using HTTPS, then you're probably using letsencrypt and so your subdomain will appear on the CT logs that are publicly accessible.

One thing you could do is use a wildcard certificate, and then use a non-obvious subdomain from that. I actually have something similar - in my set up, all my web-traffic goes to haproxy frontends which forward traffic to the appropriate backend, and I was sick of setting up multiple new certificates for each new subdomain, so I just replaced them all with a single wildcard cert instead. This means that I'm not advertising each new subdomain on the CT list, and even though they all look nominally the same when visiting - same holding page on index and same /api handling, just one of the subdomains decodes an additional URL path that provides access to status monitoring.

Separately, that Palo Alto Networks company is a real pain. They connect to absolutely everything in their attempts to spam the internet. Frankly, I'm sick of even my mail servers being bombarded with HTTP requests on port 25 and the resultant log spam.

Re: Ask HN: How did the internet discover my subdomain?

#124

Post by kelnos »

jchw wrote: post #115
lewiscollard wrote: post #106

Earlier quoted context omitted.

Depending on the web server's configuration, you very much _can_ find the domain which is configured on an IP address, by attempting to connect to that IP address via HTTPS and seeing what certificate gets served. Here's an example: https://138.68.161.203/ > Web sites prove their identity via certificates. Firefox does not trust this site because it uses a certificate that is not valid for 138.68.161.203. The certifi…

I don't think that does you any good for Cloudflare, though. They will definitely be using SNI.

That doesn't really matter, though. While OP is using Cloudflare, the actual server behind it is still a publicly-accessible IP address that an IPv4 space scanner can easily stumble upon.

Re: Ask HN: How did the internet discover my subdomain?

#125

Post by okasaki »

mr_mitm wrote: post #114
BLKNSLVR wrote: post #110

There are a number of companies, not just Palo Alto Networks, that perform various different scales of scans of the entire IPv4 space, some of them perform these scans multiple times per day. I setup a set of scripts to log all "uninvited activity" to a couple of my systems, from which I discovered a whole bunch of these scanner "security" companies. Personally, I treat them all as malicious. There are also services…

Getting the domain name from the IP address is not trivial, though. In fact, it should be impossible, if the name really hasn't been published (barring guessing attempts), so OP's question stands.

    $ host 209.216.230.207
    207.230.216.209.in-addr.arpa domain name pointer news.ycombinator.com.

Re: Ask HN: How did the internet discover my subdomain?

#126

Post by jhart99 »

venj wrote: post #118
mr_mitm wrote: post #114

Earlier quoted context omitted.

Getting the domain name from the IP address is not trivial, though. In fact, it should be impossible, if the name really hasn't been published (barring guessing attempts), so OP's question stands.

I had this issue with internal domains indexed by Google. The domains where not published anywhere by my company. They were dcanned by leakix.net which apparently scans the whole web for vulnerabilities and publishes web pages containing the domain names associated with each IP address. I guess they read them from the certificates

There is another source, SNI certs showing up on a server or load balancer during the TLS handshake. When the client tries to connect to a server using SNI without indicating the server, some will reply with a default or give a list of valid server names.

Re: Ask HN: How did the internet discover my subdomain?

#127

Post by alfiedotwtf »

melevittfl wrote: post #119
mr_mitm wrote: post #114

Earlier quoted context omitted.

Getting the domain name from the IP address is not trivial, though. In fact, it should be impossible, if the name really hasn't been published (barring guessing attempts), so OP's question stands.

The OP is misunderstanding what's happened, based on what's been posted. The OP has a server with an IP address. They're seeing GET requests in the server's logs and is assuming people have found the server's DNS name. In fact, the scanners are simply searching the IP address space and simply sending GET requests to any IP address they find. No DNS discovery needed.

Are you sure that’s the case? IP addresses != domain, so I’m getting bots are including the Host header in their requests containing the obfuscated domain.

My guess is OP is using a public DNS server that sells aggregated user requests. All it takes is one request from their machine to a public machine on the internet, and it’s now public knowledge.

Post reply on HN