Live data from Hacker News

How to build a IP geolocation database from scratch?

ipapi.is

111–120 of 186 posts

Re: How to build a IP geolocation database from scratch?

#111
post #17

A modern version of the ping-based geoip mentioned https://github.com/Ne00n/yammdb

This just links to a mmdb file that is already compiled, there isn't anything relevant to show this is a "modern" implementation of anything if the implementation isn't available.

Re: How to build a IP geolocation database from scratch?

#112

First, I am big fan of your articles even before I joined IPinfo, where we provide IP geolocation data service. Our geolocation methodology expands on the methodology you described. We utilize some of the publicly available datasets that you are using. However, the core geolocation data comes from our ping-based operation. We ping an IP address from multiple servers across the world and identify the location of the I…

That's pretty neat! You're basically using ping triangulation!

Trilateration (same technique as used for mobile network location - in addition to the GPS on the phone)

Re: How to build a IP geolocation database from scratch?

#113

Earlier quoted context omitted.

Does that actually work? That could explain an issue with a particular streaming service I use. There are currently some ongoing routing issues in BGP land and my ISP. When trying to stream, it says I’m using a proxy, so due to the incredible route my packets are taking, that might be it. What’s funny is that the only way to watch this service is to use a vpn right now.

of course it doesnt work but they gotta try clutching pearls and applying whatever pressure they can think of on these fronts

Why is this getting downvoted? It seems to me that a lot of the media-focused anti-piracy tooling is essentially a performance of toughness to make rightsholder execs comfortable. Everybody accepts you can't stop piracy entirely, and nobody's willing to say, "Fuck it, we'll compete on convenience and strong consumer relationships," so we all put up with this weird middle ground of performative DRM and the like. With only the rare occasional bit of honesty, as from Weird Al: https://sfba.social/@williampietri/110906012997848549

Re: How to build a IP geolocation database from scratch?

#114
post #73

Earlier quoted context omitted.

Thousands of people live in a zip code, while hundreds and thousands of people live in a city. We are literally giving away that data for free through our API and database. The creepiness of IP geolocation is mostly a meme. IP geolocation is mainly used in cybersecurity and marketing analytics. There are many ways to geolocate someone. I once came across a project that could estimate the country a user is from based…

And you're literally advertising this project as being helpful for targeted ads. So it's pretty clear from the get go that what you consider creepy isn't what I consider creepy. And having done enough reidentification work to scare myself, "thousands of people" might as well be a couple dozen or less. I get why you're defensive and why you think it's not creepy, but calling it a "meme" is insultingingly dismissive. J…

For sure. When people work in any industry long enough, it's easy to stop thinking about the basics. E.g., a retail butcher thinks of his work very differently than a cow or a vegan does.

When people work in advertising, they mostly forget that the core of their business is for-profit manipulation of people with little or no regard for truth or the people concerned. But I personally think that's kinda creepy, and only getting more so as it goes from broad manipulation of millions via mass media down to thousands, hundreds, or single individuals.

Re: How to build a IP geolocation database from scratch?

#115
post #50

First, I am big fan of your articles even before I joined IPinfo, where we provide IP geolocation data service. Our geolocation methodology expands on the methodology you described. We utilize some of the publicly available datasets that you are using. However, the core geolocation data comes from our ping-based operation. We ping an IP address from multiple servers across the world and identify the location of the I…

ICMP response time not useful for “locating” an anycasted address, some of which have logical location associated with them. See https://blog.cloudflare.com/icloud-private-relay/ for an example

Well, at least you can detect it is an anycast address, and mark it as such.

Re: How to build a IP geolocation database from scratch?

#116

First, I am big fan of your articles even before I joined IPinfo, where we provide IP geolocation data service. Our geolocation methodology expands on the methodology you described. We utilize some of the publicly available datasets that you are using. However, the core geolocation data comes from our ping-based operation. We ping an IP address from multiple servers across the world and identify the location of the I…

Great comment. I'm a big fan and customer of IPinfo, using your API in our login notification emails to say "You just logged in from Berlin, Germany. If this wasn't you click here." To provide country data for customers in their audit logs. And for anti-spam and fraud detection.

Re: How to build a IP geolocation database from scratch?

#117

If you don't want to do this yourself, you can actually just get Cloudflare to do it for you for free using a simple Worker since all Cloudflare requests contain approximate IP location information. You can also just send a request to my URL (Cloudflare Worker operated - so it should have global low latency): https://www.edenmaps.net/iplocation Use it for small applications, I don't mind. Just don't start sending me…

Located 100km from the Somali coast... I'm in Brussels, Belgium, thx for protecting my privacy :D

The result is [lon, lat]. You’ve most likely copied it onto Google maps, which works with [lat, lon]. Believe it or not, the industry still hasn’t come up with a standard order.

Re: How to build a IP geolocation database from scratch?

#118
post #84

Earlier quoted context omitted.

In the business of media streaming, rightholder will require that you check for vpn and proxies in addition to countries when deciding if a given viewer will be able to stream a given media.

Does that actually work? That could explain an issue with a particular streaming service I use. There are currently some ongoing routing issues in BGP land and my ISP. When trying to stream, it says I’m using a proxy, so due to the incredible route my packets are taking, that might be it. What’s funny is that the only way to watch this service is to use a vpn right now.

They probably just keep a list of known VPN server IP’s.

Re: How to build a IP geolocation database from scratch?

#119

Earlier quoted context omitted.

of course it doesnt work but they gotta try clutching pearls and applying whatever pressure they can think of on these fronts

Why is this getting downvoted? It seems to me that a lot of the media-focused anti-piracy tooling is essentially a performance of toughness to make rightsholder execs comfortable. Everybody accepts you can't stop piracy entirely, and nobody's willing to say, "Fuck it, we'll compete on convenience and strong consumer relationships," so we all put up with this weird middle ground of performative DRM and the like. With…

This is correct. Imagine in the days of yore, some two decades and change ago, when I was charged with implementing putting some music reserves "online" for streaming ...

[Harp music, progressive diagonal wave distortions through the viewport ...]

We had two layers of passwords (one to get to the webpage for the class, one when actually streaming via the client, which was RealPlayer) as well as an IP range restriction to campus (you live off campus? So sorry) because our lawyers were worried about what the RIAA's lawyers would find sufficient in the wake of a bunch of Napster-baited lawsuits launched at universities. The material itself was largely limited to snippets.

I wanted to say, "Calm down, have a martini or something. College students are just not going to go wild to download 128 kbps segments of old classical music," but alas I was not in charge.

Re: How to build a IP geolocation database from scratch?

#120

Earlier quoted context omitted.

There's several options/techniques for doing it. But just imagine you have a permanent zero overhead VPN. I don't know if that provider terminates long running calls, but the calls would stay up too regardless of tower.

Yes, I’m sure it is iOS anti-tracking and directly related to why firewall apps inside SIP my not know what is going on.

More likely to be just standard Mobile IP https://en.wikipedia.org/wiki/Mobile_IP. Fairly standard stuff, can cause some false positives around traveling (I've seen people get freaked out about stuff like "This person just logged in from their home state and then less then an hour later logged in from France!" when it was just mobile IP treating their phone as still in the US while they were in France on a trip, but their laptop connected over normal internet was seen as coming from France)
Post reply on HN