Live data from Hacker News

How to build a IP geolocation database from scratch?

ipapi.is

31–40 of 186 posts

Re: How to build a IP geolocation database from scratch?

#31
First, I am big fan of your articles even before I joined IPinfo, where we provide IP geolocation data service.

Our geolocation methodology expands on the methodology you described. We utilize some of the publicly available datasets that you are using. However, the core geolocation data comes from our ping-based operation.

We ping an IP address from multiple servers across the world and identify the location of the IP address through a process called multilateration. Pinging an IP address from one server gives us one dimension of location information meaning that based on certain parameters the IP address could be in any place within a certain radius on the globe. Then as we ping that IP from our other servers, the location information becomes more precise. After enough pings, we have a very precise IP location information that almost reaches zip code level precision with a high degree of accuracy. Currently, we have more than 600 probe servers across the world and it is expanding.

The publicly available information that you are referring to is sometimes not very reliable in providing IP location data as:

- They are often stale and not frequently updated.

- They are not precise enough to be generally useful.

- They provide location context at an large IP range level or even at organization level scale.

And last but not least, there is no verification process with these public datasets. With IPv4 trade and VPN services being more and more popular we have seen evidence that in some instances inaccurate information is being injected in these datasets. We are happy and grateful to anyone who submits IP location corrections to us but we do verify these correction submissions for that reason.

From my experience with our probe network, I can definitely say that it is far easier and cheaper to buy a server in New York than in any country in the middle of Africa. Location of an IP address greatly influences the value it can provide.

We have a free IP to Country ASN database that you can use in your project if you like.

https://ipinfo.io/developers/ip-to-country-asn-database

Re: How to build a IP geolocation database from scratch?

#32

Interesting but this isn't actually how geolocation is done, right? The ARIN/RIPE data isn't sufficiently accurate to be useful beyond country. Commercial geolocation involves correlating client IP vs known physical location e.g. from WiFi AP or mailing a package to the user. At least that's what I have been told over the decades.

I work in adtech and this is how we do geolocation. There's also device geolocation but if the user doesn't consent to sharing their GPS data with us, we just use IP address for targeting. Common provider for this is Maxmind; they ship a database that you host locally and query

Since you are in adtech: do you buy MaxMind, or roll your own? Are there any providers for US-only data, and therefore, cheaper?

Re: How to build a IP geolocation database from scratch?

#33

Earlier quoted context omitted.

I work in adtech and this is how we do geolocation. There's also device geolocation but if the user doesn't consent to sharing their GPS data with us, we just use IP address for targeting. Common provider for this is Maxmind; they ship a database that you host locally and query

Does Cloudflare have the same data as Maxmind? Because Cloudflare and Maxmind geolocate me to the exact same longitude/latitude.

CloudFlare uses Maxmind: https://developers.cloudflare.com/support/network/configurin...

Re: How to build a IP geolocation database from scratch?

#34

If you don't want to do this yourself, you can actually just get Cloudflare to do it for you for free using a simple Worker since all Cloudflare requests contain approximate IP location information. You can also just send a request to my URL (Cloudflare Worker operated - so it should have global low latency): https://www.edenmaps.net/iplocation Use it for small applications, I don't mind. Just don't start sending me…

[dead]

Re: How to build a IP geolocation database from scratch?

#35

Interesting but this isn't actually how geolocation is done, right? The ARIN/RIPE data isn't sufficiently accurate to be useful beyond country. Commercial geolocation involves correlating client IP vs known physical location e.g. from WiFi AP or mailing a package to the user. At least that's what I have been told over the decades.

I work in adtech and this is how we do geolocation. There's also device geolocation but if the user doesn't consent to sharing their GPS data with us, we just use IP address for targeting. Common provider for this is Maxmind; they ship a database that you host locally and query

[dead]

Re: How to build a IP geolocation database from scratch?

#36

If you don't want to do this yourself, you can actually just get Cloudflare to do it for you for free using a simple Worker since all Cloudflare requests contain approximate IP location information. You can also just send a request to my URL (Cloudflare Worker operated - so it should have global low latency): https://www.edenmaps.net/iplocation Use it for small applications, I don't mind. Just don't start sending me…

This is excellent!

Would you mind open sourcing the code for that?

Re: How to build a IP geolocation database from scratch?

#38

First, I am big fan of your articles even before I joined IPinfo, where we provide IP geolocation data service. Our geolocation methodology expands on the methodology you described. We utilize some of the publicly available datasets that you are using. However, the core geolocation data comes from our ping-based operation. We ping an IP address from multiple servers across the world and identify the location of the I…

Your comment is extremely interesting and what I was hoping to learn from the article (without an existing source of information, how do we determine the location of an IP address). Thank you!

Re: How to build a IP geolocation database from scratch?

#40

First, I am big fan of your articles even before I joined IPinfo, where we provide IP geolocation data service. Our geolocation methodology expands on the methodology you described. We utilize some of the publicly available datasets that you are using. However, the core geolocation data comes from our ping-based operation. We ping an IP address from multiple servers across the world and identify the location of the I…

Not gonna lie, this creeps the heck out of me.
Post reply on HN