Earlier quoted context omitted.
The user agent contains a partial answer. IP scanning doesn't give you the actual subdomain, so the question is slightly wrong or there are missing pieces.
Judging by the logs (user agents really) right now in the submission, it's hard to tell if the requests were actually for the domain (since the request headers aren't included) or just for the IP.
Ask HN: How did the internet discover my subdomain?
261–270 of 322 posts
Re: Ask HN: How did the internet discover my subdomain?
#262Earlier quoted context omitted.
I do something similar. Any hits on the default nginx vhost get logged, logs get parsed out and "repeat offenders" get put on the shitlist. I use ipset/iptables but this can also be done with fail2ban quite simply. https://nbailey.ca/post/block-scanners/
This is security theater.
Doing something like this can prevent you from showing up on Shodan.io which is used by many users/bots to find servers without running massive scans themselves.
Re: Ask HN: How did the internet discover my subdomain?
#263Hi, our company does this basically "as-a-service". The options how to find it are basically limitless. Best source is probably Certificate Transparency project as others suggested. But it does not end there, some other things that we do are things like internet crawl, domain bruteforcing on wildcard dns, dangling vhosts identification, default certs on servers (connect to IP on 443 and get default cert) and many oth…
Hi, former pentester here. If any one of your trusted clients is using a google/chromium based browser, the telemetry from that browser (webdiscovery) would reveal the existence of the subdomain in question. As others have said, security by obscurity doesn't work.
Google had a "gotcha" moment when Microsoft responded basically with "yeah we didn't steal it from Google, you had telemetry enabled"
Total shitshow
Re: Ask HN: How did the internet discover my subdomain?
#264Earlier quoted context omitted.
> If it is on DNS, it is discoverable. In the context of what OP is asking this is not true. DNS zones aren't enumerable - the only way to reliably get the complete contents of the zone is to have the SOA server approve a zone transfer and send the zone file to you. You can ask if a record in that zone exists but as a random user you can't say "hand over all records in this zone". I'd imagine that tools like Cloudfla…
Zone transfers are super interesting topic. Thanks for mentioning that. It's basically the way how to get all DNS records a DNS server has. Interestingly in some countries this is illegal and in some this is considered best practice. Generally, enabled zone transfers is considered as misconfiguration and should be disabled. We did research on that few months back and found out that 8% of all global name servers have…
Re: Ask HN: How did the internet discover my subdomain?
#265Earlier quoted context omitted.
Well, I sure hope the remainder of my URLs are safe.
Like, in: example.com/secret-id-48723487345 I hope the last bit is not leaked somehow (?) Btw, we need a "falsehoods programmers believe about URLs" ... Although there is: https://www.netmeister.org/blog/urls.html
I think the section named "Pathname" is wrong. It describes the path of an URL as if every server was Apache serving static files with its default configuration. It should describe how the path is converted into a HTTP request.
For instance, the article states that "all of these go to the same place : https://example.org https://example.org/ https://example.org// https://example.org//////////////////". That's wrong. A web client send a distinct HTTP request for each case, e.g starting with `GET // HTTP/1.1`. So the server will receive distinct paths. The assertion of "going to the same place" makes no sense in the general case.
Re: Ask HN: How did the internet discover my subdomain?
#266Hi, our company does this basically "as-a-service". The options how to find it are basically limitless. Best source is probably Certificate Transparency project as others suggested. But it does not end there, some other things that we do are things like internet crawl, domain bruteforcing on wildcard dns, dangling vhosts identification, default certs on servers (connect to IP on 443 and get default cert) and many oth…
DANE would help here: register a harmless sounding domainname whose name leaks nothing, use DNSSEC and NSEC3, and host your hidden service in a sub-domain whose name is a 63 byte long string of randomly selected ASCII characters. But this isn't really an option.
Re: Ask HN: How did the internet discover my subdomain?
#267Earlier quoted context omitted.
"Security by obscurity does not work" This is one of those false voyeur OS internet tennets designed to get people to publish their stuff. Obscurity is a fine strategy, if you don't post your source that's good. If you post your source, that's a risk. The fact that you can't rely on that security measure is just a basic security tennet that applies to everything: don't rely on a single security measure, use redundant…
> Subdomains can be passwords and a well crafted subdomain should not leak, I disagree. A subdomain is not secret in any way. There are many ways in which it is transmitted unencrypted. A couple: - DNS resolution, multiple resolvers and authoritative servers - TLS SNI - HTTP Host Header There are many middle boxes that could perform safety checks on behalf of the client, and drop it into a list to be rescanned. - Vir…
Agree.
But who said that all passwords or shiboleths should all be encrypted in transit?
It can serve as a canary for someone snooping your traffic. Even if you encrypt it, you don't want people snooping.
To date of my subdomains that I never publish, I haven't had anyone attempting to connect with them.
It's one of those redundant measures.
And it's also one of those risks that you take, you can maximize security by staying at home all day, but going out to take the trash is a calculated risk that you must take or risk overfocusing on security.
It's similar to port knocking. If you are encrypting it, it's counterproductive, it's a low effort finishing touch, like a nice knot.
Re: Ask HN: How did the internet discover my subdomain?
#268Earlier quoted context omitted.
I sadly did not see the comment above, but I'd like to just add, that this bruteforce and sniffing methods are target only against our paying customers. We built global reverse-DNS dataset solely from cert transparency logs. Our active scanning/bruteforcing runs only for assets owned by our customers.
…as long as your tools are only in your hands to be used, correct? Once a tool is created and used on a machine with access to the greater internet, doesn’t your logic hold that its security is compromised inherently? Not saying you have been infiltrated, or a rogue employee has cleverly exported a copy or the methodology to duplicate it off-site, but I’m not saying that hasn’t happened either.
Re: Ask HN: How did the internet discover my subdomain?
#269Earlier quoted context omitted.
"Security by obscurity does not work" This is one of those false voyeur OS internet tennets designed to get people to publish their stuff. Obscurity is a fine strategy, if you don't post your source that's good. If you post your source, that's a risk. The fact that you can't rely on that security measure is just a basic security tennet that applies to everything: don't rely on a single security measure, use redundant…
> "Security by obscurity does not work" The saying is "security by obscurity is not security" which is absolutely true. If your security relies on the attacker not finding it or not knowing how it works, it's not actually secure. Obscurity has its own value of course, I strongly recommend running any service that's likely to be scanned for regularly on non-standard ports wherever practical simply to reduce the number…
Again, if your security relies on any one thing, it's a problem. A secure system needs redundant mechanisms.
Can you think of a single mechanism that if implemented would make a system secure? I think not.
Re: Ask HN: How did the internet discover my subdomain?
#270Hi, our company does this basically "as-a-service". The options how to find it are basically limitless. Best source is probably Certificate Transparency project as others suggested. But it does not end there, some other things that we do are things like internet crawl, domain bruteforcing on wildcard dns, dangling vhosts identification, default certs on servers (connect to IP on 443 and get default cert) and many oth…
"Security by obscurity does not work" This is one of those false voyeur OS internet tennets designed to get people to publish their stuff. Obscurity is a fine strategy, if you don't post your source that's good. If you post your source, that's a risk. The fact that you can't rely on that security measure is just a basic security tennet that applies to everything: don't rely on a single security measure, use redundant…
People consistently misuse the Swiss cheese security metaphor to justify putting multiple ineffective security barriers in place.
The holes in the cheese are supposed to represent unknown or very difficult to exploit flaws in your security layers, and that's why you ideally want multiple layers.
You can't just stack up multiple known to be broken layers and call something secure. The extra layers are inconvenient to users and readily bypassed by attackers by simply tackling them one at a time.
Security by obscurity is one such layer.