Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to build a IP geolocation database from scratch?

ipapi.is

Post reply on HN
Print view View original

161–170 of 186 posts

Re: How to build a IP geolocation database from scratch?

#161

Post by meroje »

withinboredom wrote: post #85
meroje wrote: post #84

Earlier quoted context omitted.

In the business of media streaming, rightholder will require that you check for vpn and proxies in addition to countries when deciding if a given viewer will be able to stream a given media.

Does that actually work? That could explain an issue with a particular streaming service I use. There are currently some ongoing routing issues in BGP land and my ISP. When trying to stream, it says I’m using a proxy, so due to the incredible route my packets are taking, that might be it. What’s funny is that the only way to watch this service is to use a vpn right now.

Routing should not impact the detection, it's usually based on maxmind's anonymous/datacenter database using your IP. Accuracy won't be 100% of course but you have to show compliance.

Re: How to build a IP geolocation database from scratch?

#162

Post by reincoder »

yrro wrote: post #160
reincoder wrote: post #31

First, I am big fan of your articles even before I joined IPinfo, where we provide IP geolocation data service. Our geolocation methodology expands on the methodology you described. We utilize some of the publicly available datasets that you are using. However, the core geolocation data comes from our ping-based operation. We ping an IP address from multiple servers across the world and identify the location of the I…

hm, ipinfo.io tells me that I'm using a VPN even though I'm not...

Our VPN recognition is behavior-based. So, there is probably a chance that the IP address you are using is showing some of those behavior patterns.

A behavior pattern could be that your IP address is being shuffled around random locations that go beyond the normal location shuffling of an ISP connection.

Also, if your IP range is listed in some public datasets that belong to a VPN service, we could recognize your IP as a VPN.

Please reach out to our support and let us know about this. Thanks

Re: How to build a IP geolocation database from scratch?

#163

Post by sgjohnson »

vinay_ys wrote: post #28
mootothemax wrote: post #8

Any suggestions for geolocating datacenter IPs, even very roughly? I'm analysing traceroute data, and while I have known start and end locations, it's the bit in the middle I'm interested in. I can infer certain details from airport codes in node hostnames, for example. It would also be possible - I guess - to infer locations based on average RTT times, presuming a given node's not having a bad day. Anyone have any o…

> A couple of troublesome example IPs are 193.142.125.129, 129.250.6.113, and 129.250.3.250. They come up in a UK traceroute - and I believe they're in London - but geolocate all over the world. If I'm running a popular app/web service, I would have my own AS number and I will have purchased a few blocks of IP addresses under this AS and then I would advertize these addresses from multiple owned/rented datacenters ar…

> If you have ability to do traceroute from multiple probes sprinkled across the globe with known locations

Everyone who's aware of RIPE Atlas has that ability.

I have almost a billion RIPE Atlas credits. A single traceroute costs 60. I have enough credits to run several traceroutes on the entire IPv4 internet. (the smallest possible BGP announcement is /24, so max of 2^24 traceroutes, but in reality it's even less).

Re: How to build a IP geolocation database from scratch?

#164

Post by sgjohnson »

eptyc1 wrote: post #49
kube-system wrote: post #44

Earlier quoted context omitted.

iptables -A INPUT -p icmp -j DROP

Indeed. Openwrt for some reason defaults to reply to pings. I see the value of ICMP for servers, but I don't see the value for home ISP routers. I disabled ICMP reply on my home router.

> I disabled ICMP reply on my home router.

Doesn't actually help at all because the BGP announced prefix of your IP can still be tracerouted. You won't be physically far from it.

Say if your ISP announces 125.15.18.0/17 and you're in 125.15.29.145, a traceroute will still yield a pretty good approximation of where you're at. The last hop ping is really quite immaterial here.

Re: How to build a IP geolocation database from scratch?

#165

Post by sgjohnson »

kube-system wrote: post #75
chaps wrote: post #48

Earlier quoted context omitted.

Cool. Thanks. But let's say I do.

Then there's nothing you can do. If you respond to pings, then others can take note of the responses you send.

And if you don't respond to pings, a traceroute can still be used to find the hop before yours, which will almost certainly achieve the same result for geolocation purposes.

Re: How to build a IP geolocation database from scratch?

#166

Post by quectophoton »

jl6 wrote: post #37

Is anybody maintaining a historical archive of “IP address metadata” (which would include geolocation)? If I have logs from 10 years ago, can I look up information about that IP as it was at the time?

Maybe not exactly what you're looking for, but the ipfire project has a git repository[1] mapping address ranges to countries. It apparently going back to 2017 only.

[1]: https://git.ipfire.org/?p=location/location-database.git;a=s...

Re: How to build a IP geolocation database from scratch?

#167

Post by fer »

junto wrote: post #135

As someone that lives in a country where the national language is not my first language, I hate websites that use IP location to make assumptions about my choice of language and it being forced on me based on a lazy assumption, when my browser is sending language headers quite clearly, and they are ignored.

I share your frustration so much that I wrote an article about it: https://www.fer.xyz/2021/04/i18n

Re: How to build a IP geolocation database from scratch?

#168

Post by withinboredom »

meroje wrote: post #161
withinboredom wrote: post #85

Earlier quoted context omitted.

Does that actually work? That could explain an issue with a particular streaming service I use. There are currently some ongoing routing issues in BGP land and my ISP. When trying to stream, it says I’m using a proxy, so due to the incredible route my packets are taking, that might be it. What’s funny is that the only way to watch this service is to use a vpn right now.

Routing should not impact the detection, it's usually based on maxmind's anonymous/datacenter database using your IP. Accuracy won't be 100% of course but you have to show compliance.

I doubt it. According to that database my ip is in a totally different country but I'm served the correct content. Despite my efforts to fix this for years...

Re: How to build a IP geolocation database from scratch?

#169

Post by mrklol »

louison11 wrote: post #117
hotgeart wrote: post #87

Earlier quoted context omitted.

Located 100km from the Somali coast... I'm in Brussels, Belgium, thx for protecting my privacy :D

The result is [lon, lat]. You’ve most likely copied it onto Google maps, which works with [lat, lon]. Believe it or not, the industry still hasn’t come up with a standard order.

You could also just return a json with lat and lon as fields so nobody is getting confused. {lat:5, lon:7}
Post reply on HN