Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to build a IP geolocation database from scratch?

ipapi.is

Post reply on HN
Print view View original

1–10 of 186 posts

Re: How to build a IP geolocation database from scratch?

#5

Post by incolumitas »

TZubiri wrote: post #3

"how to scrape an ip geolocation database" You know you can just run a whois query per ip you want to analyze, no point in scraping the whole ipvN space.

I have to scrape the whole IP address space since I offer location information as part of my API.

Also I only need to scrape as many WHOIS records as there are different networks out there. So for example for the IPv4 address space, there are much less networks as there are IPv4 addresses (2^32).

Also, most RIR's provide their WHOIS databases for download.

Therefore, "scraping" is not really the correct word, it's an hybrid approach, but mostly based on publicly available data from the five RIR's.

Re: How to build a IP geolocation database from scratch?

#7

Post by notlukesky »

incolumitas wrote: post #5
TZubiri wrote: post #3

"how to scrape an ip geolocation database" You know you can just run a whois query per ip you want to analyze, no point in scraping the whole ipvN space.

I have to scrape the whole IP address space since I offer location information as part of my API. Also I only need to scrape as many WHOIS records as there are different networks out there. So for example for the IPv4 address space, there are much less networks as there are IPv4 addresses (2^32). Also, most RIR's provide their WHOIS databases for download. Therefore, "scraping" is not really the correct word, it's an…

What was the easiest and the most frustrating part?

Re: How to build a IP geolocation database from scratch?

#8

Post by mootothemax »

Any suggestions for geolocating datacenter IPs, even very roughly? I'm analysing traceroute data, and while I have known start and end locations, it's the bit in the middle I'm interested in.

I can infer certain details from airport codes in node hostnames, for example.

It would also be possible - I guess - to infer locations based on average RTT times, presuming a given node's not having a bad day.

Anyone have any other ideas?

Edit: A couple of troublesome example IPs are 193.142.125.129, 129.250.6.113, and 129.250.3.250. They come up in a UK traceroute - and I believe they're in London - but geolocate all over the world.

Post reply on HN