Live data from Hacker News

Find every domain someone owns automatically

securitytrails.com

71–80 of 110 posts

Re: Find every domain someone owns automatically

#72
post #59

How do you plan on handling the EU’s “right to be forgotten” (it’s pretty straightforward to make the argument you’re a search engine) and other components of the GDPR?

"The right to erasure" is not an absolute right for anyone to get all their data deleted. If the data owner (read: the registrars) still have a legal right to collect and maintain the data public and it has not been revoked one could argue that they (security trails) don't have to remove the data. It's my understanding that the registrars are the ones with the burden here. They need to inform everyone of the data era…

>one could argue that they (security trails) don't have to remove the data.

It doesn't work that way. The "right to be forgotten" can be used to remove search results from Google, even if the original content stays up.

Re: Find every domain someone owns automatically

#73
Is it possible to include WHOIS data from way back when InterNIC was the only registrar?

I own a domain I registered way back when you would send an email to InterNIC and registration was free. The WHOIS data returned for the domain only starts in 2008 and skips about 12-14 years.

Re: Find every domain someone owns automatically

#74

I didn't look into what this is doing but it's not finding 1/2 of my domains (they are not private). I'm guessing it's not "Find every domain someone owns" it's "Find every domain that meets X criteria" which may more may not be every domain someone owns.

Yeah, I was fairly unimpressed when it didn't even manage to lookup the first domain I entered (it's not new and it's not private). I guess it might be useful for some domains, but there's no way I'd rely on it.

Re: Find every domain someone owns automatically

#75

Just tried it with some of my own details. Found a domain I forgot I owned on an old reseller I havent used in a while - lol. On a more serious note - I'm very curious how you get such a long history of domains. i.e. I can see every DNS change and ownership for any domain - I didnt realise that was always available?

Hi! We acquired 4 companies that have been doing lots of cool data work. We also license and collect our own data to mix in. There's different granularity depending on the data (WHOIS history, Name Server history, DNS record history, technologies used) etc. We're constantly improving it. The thesis we have is that if you get hacked, it often times is through an old server or satellite domain. We're building tools to…

Just out of curiosity, was one of those companies originally named DeletedDomains.com? They had the full root zone (of the ones now managed by Verisign GRS anyway) since approximately 2001 (my involvement with them was 2003).

Re: Find every domain someone owns automatically

#76

Hmm is there a technical write up of how you are pulling this data? I tried one of our companies for fun and it’s only pulling 1.4million records in one place and then 65,000 in another. Doesn’t seem to have all our nameservers or relays either.

Based on the supported TLDs, I’m guessing they are pulling down the root zone tables from Verisign GRS. Verisign licenses the Whois data in bulk out to companies like theirs.

Re: Find every domain someone owns automatically

#78
post #9

Earlier quoted context omitted.

It's pretty much missing or empty data for the entire .au TLD (e.g. even google.com.au is not present). Living here, I know that their whois server rate limits heavily and the ccTLD zonefile is not available, so I'm guessing those are probably contributing reasons. Do you guys do any crawling at all? I think this is a fantastic concept though. Knowing that whois database and zonefile access is often protected for com…

right now we're getting .au domains from Open Crawl. It's tricky because the zone file is not available like you mentioned. If anyone has any ideas on how to get more .au domains, we'll gladly implement.

There's some reverse engineered zone files for a lot of cctlds at http://viewdns.info/data/.
Post reply on HN