Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How did the internet discover my subdomain?

news.ycombinator.com

Post reply on HN
Print view View original

91–100 of 322 posts

Re: Ask HN: How did the internet discover my subdomain?

#91

Post by p0w3n3d »

paxys wrote: post #62

Not sure why everyone is going on about certificate transparency logs when the answer is right there in the user agent. The company is scanning the ipv4 space and came upon your IP and port.

Finding IP does not mean finding the domain. When doing HTTP request to IP you specify the domain you want to connect to. For example you can configure your /etc/hosts to have xxxnakedhamsters.google.com pointing to 8.8.8.8 and make the http request, which will cause Google getting the domain request (i.e. header Host: xxxnakedhamsters.google.com) and it will refuse it or try to redirect to http. Of course it's only related to HTTP because HTTPS will require certificate. That's why they're speaking about certificates.

Re: Ask HN: How did the internet discover my subdomain?

#92

Post by majke »

walrus01 wrote: post #89
alexjplant wrote: post #27

Earlier quoted context omitted.

> If it is on DNS, it is discoverable. In the context of what OP is asking this is not true. DNS zones aren't enumerable - the only way to reliably get the complete contents of the zone is to have the SOA server approve a zone transfer and send the zone file to you. You can ask if a record in that zone exists but as a random user you can't say "hand over all records in this zone". I'd imagine that tools like Cloudfla…

> In the context of what OP is asking this is not true. DNS zones aren't enumerable - the only way to reliably get the complete contents of the zone is to have the SOA server approve a zone transfer and send the zone file to you. This is generally true but also if you watch authoritative-only dns server logs for text strings matching ACL rejections, there's plenty of things out there which are fully automated crawler…

In the context of DNSSEC dns zones are very much enumerable. Cloudflare does amazing tricks to avoid this https://blog.cloudflare.com/black-lies/

Re: Ask HN: How did the internet discover my subdomain?

#93

Post by fragmede »

peeters wrote: post #68
paxys wrote: post #62

Not sure why everyone is going on about certificate transparency logs when the answer is right there in the user agent. The company is scanning the ipv4 space and came upon your IP and port.

It's rather hilarious that nobody mentioned this in 7 hours. What am I missing? ~5 billion scans in a few hours is nothing for a company with decent resources. OP: in case you didn't follow, they're literally trying every possible IPv4 address and seeing if something exists on standard ports at that address. I believe it would be harder to find out your domain that way if you were using SNI and only forwarded/served…

Just the default hostname. It won't reveal all of them or any of the IP addresses of that box. secret-freedom-fighter.ice-cream-shop.example.com could have the same IP as example.com and you'd only know example.com

Re: Ask HN: How did the internet discover my subdomain?

#94

Post by A1kmm »

fragmede wrote: post #93
peeters wrote: post #68

Earlier quoted context omitted.

It's rather hilarious that nobody mentioned this in 7 hours. What am I missing? ~5 billion scans in a few hours is nothing for a company with decent resources. OP: in case you didn't follow, they're literally trying every possible IPv4 address and seeing if something exists on standard ports at that address. I believe it would be harder to find out your domain that way if you were using SNI and only forwarded/served…

Just the default hostname. It won't reveal all of them or any of the IP addresses of that box. secret-freedom-fighter.ice-cream-shop.example.com could have the same IP as example.com and you'd only know example.com

If you've got one cert with a subject alt name for each host, they'd see them all. If you use SNI and they have different certificates, the domains might still be in Certificate Transparency logs. If a wildcard cert is used, that could help to conceal the exact subdomain.

Re: Ask HN: How did the internet discover my subdomain?

#95

Post by fc417fc802 »

snailmailman wrote: post #65
thedougd wrote: post #53

Some CAs (Amazon) allow not publishing to the Certificate Transparency Log. But if you do this, browsers will block the connection by default. Chromium browsers have a policy option to skip this check for selected URLs. See: CertificateTransparencyEnforcementDisabledForURLs. Some may find this more desirable than wildcard certificates and their drawbacks.

Firefox is currently rolling out the same thing. They will treat any non-publicly-logged certificate as insecure. I’m surprised amazon offers the option to not log certificates. The whole idea is that every issued cert should get logged. That way, fraudulently-issued certs are either well documented in public logs- or at least not trusted by the browser.

It doesn't seem like the choice has any impact on that. It just protects user privacy if that's what they want to prioritize.

Depending on the issuer logging all certs would never work. You can't rely on the untrusted entity to out themselves for you.

The security comes from the browser querying the log and warning you if the entry is missing. In that sense declining to log a cert is similar to self signing one. The browser will warn and users will need to accept. As long as the vast majority of sites don't do that then we maintain a sort of herd immunity because the warnings are unexpected by the end user.

Re: Ask HN: How did the internet discover my subdomain?

#96

Post by fc417fc802 »

arccy wrote: post #17
vince14 wrote: post #13

I'm having the same issue. https://securitytrails.com/ also had my "secret" staging subdomain. I made a catch-all certificate, so the subdomain didn't show up in CT logs. It's still a secret to me how my subdomain ended up in their database.

maybe your server responded to a plain ip addressed request with the real name...

He said he used a wildcard cert though. So what part of the response would contain the subdomain in that case?

Re: Ask HN: How did the internet discover my subdomain?

#97

Post by alexjplant »

fulafel wrote: post #55
alexjplant wrote: post #27

Earlier quoted context omitted.

> If it is on DNS, it is discoverable. In the context of what OP is asking this is not true. DNS zones aren't enumerable - the only way to reliably get the complete contents of the zone is to have the SOA server approve a zone transfer and send the zone file to you. You can ask if a record in that zone exists but as a random user you can't say "hand over all records in this zone". I'd imagine that tools like Cloudfla…

In practice it's not so far fetched: A zone transfer is just another dns query at the protocol level, i suppose you can conceptually view it as sending a file if you consider the dns response a file. Something like "host -t axfr my.domain ns1.my.domain" will show the zone depending on how a domain's name server is configured (eg in bind, allow-transfer directive can be used to make it public, require ip acl to match…

> in bind, allow-transfer directive

Configuring BIND as an authoritative server for a corporate domain when I was a wee lad is how I learned DNS. It was and still is bad practice to allow zone transfers without auth. If memory serves I locked it down between servers via key pairs.

Re: Ask HN: How did the internet discover my subdomain?

#98

Post by ghusto »

p0w3n3d wrote: post #91
paxys wrote: post #62

Not sure why everyone is going on about certificate transparency logs when the answer is right there in the user agent. The company is scanning the ipv4 space and came upon your IP and port.

Finding IP does not mean finding the domain. When doing HTTP request to IP you specify the domain you want to connect to. For example you can configure your /etc/hosts to have xxxnakedhamsters.google.com pointing to 8.8.8.8 and make the http request, which will cause Google getting the domain request (i.e. header Host: xxxnakedhamsters.google.com) and it will refuse it or try to redirect to http. Of course it's only…

First thing I’d do for an IP that answers is a reverse lookup, so I expect that’s at least in the list of things they’d try.

Re: Ask HN: How did the internet discover my subdomain?

#99

Post by opello »

arkfil wrote: post #80

paloAlto (network devices like firewalls etc) is able to scan the sites that users want to visit behind their devices. these are very popular devices in many companies. users can also have agents installed on their computers that also have access to the sites they visit.

This is what I was thinking it must be, along the lines of Cisco NAC. Could monitor via browser plugin for full URLs or DNS server for domains.

I imagine the certificate transparency log is the avenue, but local monitoring and reporting up as a new URL or domain to scan for malware seems similarly plausible.

Re: Ask HN: How did the internet discover my subdomain?

#100

Post by fc417fc802 »

johnklos wrote: post #28
vince14 wrote: post #13

I'm having the same issue. https://securitytrails.com/ also had my "secret" staging subdomain. I made a catch-all certificate, so the subdomain didn't show up in CT logs. It's still a secret to me how my subdomain ended up in their database.

Serious question: Do you really think that Cloudflare is trying to keep these kinds of thing private? If so, I'd suggest that's not a reasonable expectation.

Related question (not rhetorical). If you do DNS for subdomains yourself (and just use Cloudflare to point dns.example.com at your box) will the subdomain queries leak and show up in aggregate datasets? What I'm asking is if query recursion is always handled locally or if any of the reasonably common software stacks resolve it remotely.
Post reply on HN