Find every domain someone owns automatically • Hacker News Board

Re: Find every domain someone owns automatically

#11

Post by mxpxrocks10 » Sun, Dec 03, 2017, 4:36 AM UTC

regecks wrote: post #9

mxpxrocks10 wrote: post #4

Earlier quoted context omitted.

Hey, we're still kind of new and kicking the tires. Any UX feedback or data feedback is appreciated!

It's pretty much missing or empty data for the entire .au TLD (e.g. even google.com.au is not present). Living here, I know that their whois server rate limits heavily and the ccTLD zonefile is not available, so I'm guessing those are probably contributing reasons. Do you guys do any crawling at all? I think this is a fantastic concept though. Knowing that whois database and zonefile access is often protected for com…

right now we're getting .au domains from Open Crawl. It's tricky because the zone file is not available like you mentioned. If anyone has any ideas on how to get more .au domains, we'll gladly implement.

Re: Find every domain someone owns automatically

#12

Post by wiradikusuma » Sun, Dec 03, 2017, 4:45 AM UTC

It doesn't work for privacy-protected whois (obviously), but it also doesn't work for .id domains (try http://every.id, http://awesome.id or http://player.id).

Re: Find every domain someone owns automatically

#13

Post by mxpxrocks10 » Sun, Dec 03, 2017, 4:47 AM UTC

wiradikusuma wrote: post #12

It doesn't work for privacy-protected whois (obviously), but it also doesn't work for .id domains (try http://every.id , http://awesome.id or http://player.id ).

We'll add .id to the wish list. Tricky because they don't make the zone file avail, but we'll figure something out. Thanks for the note.

Re: Find every domain someone owns automatically

#14

Post by arca_vorago » Sun, Dec 03, 2017, 4:50 AM UTC

mxpxrocks10 wrote: post #10

jgh wrote: post #8

i guess this doesn't work for private registration. E.g. Domains by Proxy's phone number has ~11 million domains.

We have a really fun blog post coming out for this next week or so. One of our team found a pretty big bug in certain private registrations. Stay tuned.

You say fun, but in the era of swatting and doxxing, private registrations are often the first line of defense for those who don't have a USPS or ups mail address if they don't have a physical office/address. I would be very careful about how you responsibly disclose any such bugs.

Re: Find every domain someone owns automatically

#15

Post by ploggingdev » Sun, Dec 03, 2017, 4:52 AM UTC

mxpxrocks10 wrote: post #10

jgh wrote: post #8

i guess this doesn't work for private registration. E.g. Domains by Proxy's phone number has ~11 million domains.

We have a really fun blog post coming out for this next week or so. One of our team found a pretty big bug in certain private registrations. Stay tuned.

Can you give a gist of your findings?

I'll take a guess : customers are given unique whois protected email addresses allowing you to find all the domains owned by the person. Eg- If I own abc.com and xyz.com, both have the same public email address. The problem with such bugs is that there's no way to undo the damage since historical whois records are archived.

Re: Find every domain someone owns automatically

#16

Post by mxpxrocks10 » Sun, Dec 03, 2017, 5:04 AM UTC

arca_vorago wrote: post #14

mxpxrocks10 wrote: post #10

Earlier quoted context omitted.

We have a really fun blog post coming out for this next week or so. One of our team found a pretty big bug in certain private registrations. Stay tuned.

You say fun, but in the era of swatting and doxxing, private registrations are often the first line of defense for those who don't have a USPS or ups mail address if they don't have a physical office/address. I would be very careful about how you responsibly disclose any such bugs.

Good point. Poor word choice on my end. Thanks for the note. The stuff we found is not quite that abrasive but very interesting. How would you recommend disclosing?

Re: Find every domain someone owns automatically

#17

Post by moonka » Sun, Dec 03, 2017, 5:07 AM UTC

mxpxrocks10 wrote: post #5

moonka wrote: post #2

That is very handy.

woot- glad you like it so far. Any actionable feedback greatly appreciated.

It worked exactly as I hoped. I couldn't figure out why some domains I were expecting to show up weren't, but searching by email reminded me that I need up update some of my whois info due to a name change.

Re: Find every domain someone owns automatically

#18

Post by mxpxrocks10 » Sun, Dec 03, 2017, 5:08 AM UTC

moonka wrote: post #17

mxpxrocks10 wrote: post #5

Earlier quoted context omitted.

woot- glad you like it so far. Any actionable feedback greatly appreciated.

It worked exactly as I hoped. I couldn't figure out why some domains I were expecting to show up weren't, but searching by email reminded me that I need up update some of my whois info due to a name change.

awesome! any feedback or ideas are appreciated, thanks for trying it out.

Re: Find every domain someone owns automatically

#19

Post by kuschku » Sun, Dec 03, 2017, 5:09 AM UTC

So, for .de it just returns either NULL, undefined, or empty for everything, for .eu it errors out entirely, and for the domains of me it does find, it has wrong data.

kuschku.de has, for the past 2 years, always pointed at 51.15.1.223 or 163.172.217.134, never at 204.236.227.242. Funnily, for other domains pointed at the same IPs, it has correct data – e.g. quasseldroid.info correctly shows the IP history.

The datasets used here seem of questionable quality, souring the taste of this awesome feature.

Re: Find every domain someone owns automatically

#20

Post by mxpxrocks10 » Sun, Dec 03, 2017, 5:12 AM UTC

kuschku wrote: post #19

So, for .de it just returns either NULL, undefined, or empty for everything, for .eu it errors out entirely, and for the domains of me it does find, it has wrong data. kuschku.de has, for the past 2 years, always pointed at 51.15.1.223 or 163.172.217.134, never at 204.236.227.242. Funnily, for other domains pointed at the same IPs, it has correct data – e.g. quasseldroid.info correctly shows the IP history. The datas…

hey- would love it if you can drop me an email (in profile) so we can debug/improve.

.DE is particularly hard because they lock down the zone file. GTLDs like .INFO are easier to get because the zone files are open. We have 9 years daily granularity for the gTLDs.

Would love to clean up the U/X so it's clear what we have data for and what we don't to be completely transparent.

Thanks for the note.