Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to build a IP geolocation database from scratch?

ipapi.is

Post reply on HN
Print view View original

81–90 of 186 posts

Re: How to build a IP geolocation database from scratch?

#81

Post by freedomben »

reincoder wrote: post #43
TheClassic wrote: post #38

Earlier quoted context omitted.

Your comment is extremely interesting and what I was hoping to learn from the article (without an existing source of information, how do we determine the location of an IP address). Thank you!

I really appreciate. Thank you. We are very transparent about our process. If you have any questions, you can always reach out to us. We have a simplified explanation of our probe network here: https://ipinfo.io/blog/probe-network-how-we-make-sure-our-da... The only update is the number of servers is like 600+ now. The probe network is growing extremely rapidly. Our IP geolocation process is quite complicated, and we…

Just wanted to let you know, it's this transparency that turned me into a customer!

I love your company and service, but I hate your pricing. I work with a lot of small clients/apps that paying for usage would be a no-brainer, but the defined monthly price buckets don't make any economical sense at their scale. If you added a "pay as you go" tier that a small app could reasonably start by using dollars worth of API calls per month and grow from there, I'd be spreading your seed all over the place. I'm not saying this to rag on you, just trying to provide some constructive feedback as a thank you for your info sharing!

Re: How to build a IP geolocation database from scratch?

#82

Post by Daviey »

kam wrote: post #80
Daviey wrote: post #77

Earlier quoted context omitted.

Would you consider no-signup inspection of the data you hold on the requesters IP address? I would love to see what you have on MY IP address, and if sufficiency accurate it feels that it would be a good incentive to sign up to use commerically. It feels like it couldn't be abused by 'freeloaders', because i'd guess their use-case is viewing other peoples.

This is literally the most prominent thing on the https://ipinfo.io home page.

That's embarrassing for me... I thought that was a static image of an example. And I did look through the site looking for a search. Oops.

Re: How to build a IP geolocation database from scratch?

#83

Post by kube-system »

chaps wrote: post #78
kube-system wrote: post #75

Earlier quoted context omitted.

Then there's nothing you can do. If you respond to pings, then others can take note of the responses you send.

You're missing the point that the question is effectively asking for a list of hosts that they can block. Edit: they provided a method: https://news.ycombinator.com/item?id=37510063

I understand that was the initial question. I am saying that is a fools errand. Anyone with a few VPSes, a calculator, and a map can do this. It isn't just ipinfo.io doing this. There are a lot of ip geolocation services.

Re: How to build a IP geolocation database from scratch?

#84

Post by meroje »

withinboredom wrote: post #59
incolumitas wrote: post #54

Earlier quoted context omitted.

Big fan of what articles? On https://incolumitas.com/ or on https://ipapi.is/ ? Great idea with latency triangulation, I used latency information for a lot of things, especially VPN and Proxy detection. But I didn't assume you can obtain that accurate location. I am honestly impressed. But latency triangulation with 600 servers gives some very good approximation. Nice man! Some questions: - ICMP traffic is penalised/…

I'm very curious why you'd do VPN/proxy detection... But at a previous company I worked at that ran a very large chunk of the internet, we did indexing of nearly the entire internet (even large portions of the dark web) approximately every two weeks. There were about 500 servers doing that non-stop. So, I think it is relatively reasonable if you have 600 servers to do that.

In the business of media streaming, rightholder will require that you check for vpn and proxies in addition to countries when deciding if a given viewer will be able to stream a given media.

Re: How to build a IP geolocation database from scratch?

#85

Post by withinboredom »

meroje wrote: post #84
withinboredom wrote: post #59

Earlier quoted context omitted.

I'm very curious why you'd do VPN/proxy detection... But at a previous company I worked at that ran a very large chunk of the internet, we did indexing of nearly the entire internet (even large portions of the dark web) approximately every two weeks. There were about 500 servers doing that non-stop. So, I think it is relatively reasonable if you have 600 servers to do that.

In the business of media streaming, rightholder will require that you check for vpn and proxies in addition to countries when deciding if a given viewer will be able to stream a given media.

Does that actually work? That could explain an issue with a particular streaming service I use. There are currently some ongoing routing issues in BGP land and my ISP. When trying to stream, it says I’m using a proxy, so due to the incredible route my packets are taking, that might be it. What’s funny is that the only way to watch this service is to use a vpn right now.

Re: How to build a IP geolocation database from scratch?

#86

Post by inemesitaffia »

detourdog wrote: post #79
reincoder wrote: post #31

First, I am big fan of your articles even before I joined IPinfo, where we provide IP geolocation data service. Our geolocation methodology expands on the methodology you described. We utilize some of the publicly available datasets that you are using. However, the core geolocation data comes from our ping-based operation. We ping an IP address from multiple servers across the world and identify the location of the I…

I just noticed that my wifes iphone uses the same mycingular ip address while driving accross 3 states over 5 hours.l while checking mail.

There's several options/techniques for doing it. But just imagine you have a permanent zero overhead VPN.

I don't know if that provider terminates long running calls, but the calls would stay up too regardless of tower.

Re: How to build a IP geolocation database from scratch?

#87

Post by hotgeart »

louison11 wrote: post #11

If you don't want to do this yourself, you can actually just get Cloudflare to do it for you for free using a simple Worker since all Cloudflare requests contain approximate IP location information. You can also just send a request to my URL (Cloudflare Worker operated - so it should have global low latency): https://www.edenmaps.net/iplocation Use it for small applications, I don't mind. Just don't start sending me…

Located 100km from the Somali coast... I'm in Brussels, Belgium, thx for protecting my privacy :D

Re: How to build a IP geolocation database from scratch?

#88

Post by reincoder »

chaps wrote: post #73
reincoder wrote: post #62

Earlier quoted context omitted.

Thousands of people live in a zip code, while hundreds and thousands of people live in a city. We are literally giving away that data for free through our API and database. The creepiness of IP geolocation is mostly a meme. IP geolocation is mainly used in cybersecurity and marketing analytics. There are many ways to geolocate someone. I once came across a project that could estimate the country a user is from based…

And you're literally advertising this project as being helpful for targeted ads. So it's pretty clear from the get go that what you consider creepy isn't what I consider creepy. And having done enough reidentification work to scare myself, "thousands of people" might as well be a couple dozen or less. I get why you're defensive and why you think it's not creepy, but calling it a "meme" is insultingingly dismissive. J…

I grok and understand your concern. I am not being defensive; I am just trying to provide an explanation. I really enjoy having conversations like this with developers as honestly and empathetically possible.

I apologize if I was rude in any way by saying the word "meme". I saw a sister comment and thought you were being sarcastic. There is a popular meme about "I have your IP address", so I thought you were referencing that. I have had conversations with many young people who were concerned about their IP address being leaked through a game server. Therefore, I try to use humor to alleviate their stress. However, I now realize that this situation was different, and I am sorry for not understanding that.

We provide a service that helps users keep their internet-connected services secure by providing IP metadata information. Are you being attacked by malicious actors? Use our free IP database to identify the location and ASN to block them. Do you want to restrict access to your service to certain regions? Do that for free with our services.

We have the most accurate data available, and yet we offer the most generous free tier. We provide a full accuracy IP database for free, without any range aggregation, and with daily updates and a commercially permissible license. We have built a community forum solely dedicated to answering users' questions. We invest in website tools and open-source tools, all with the goal of helping users maintain the security and functionality of their services.

We do have premium tier services, but if you use our free data as a foundation, you can always replicate those premium features to a reliable degree.

Our IP metadata information is being used in marketing and sales intelligence. It is the same data that you use to protect your internet connected devices, used by our customers to sell you something.

IP metadata information that we provide is a cornerstone of keeping the internet safe and accessible for everyone. That is how things just are. The deepweb is immune to IP meta data information, and that is why it is such a messy and chaotic place.

That is just truth of the internet. We are essential and we prefer to be open about our process and listen to our stakeholders (users + customers + non-users).

Re: How to build a IP geolocation database from scratch?

#89

Post by jedberg »

It all depends on what you want to use it for and how accurate it needs to be.

The best way to build a geolocation service is to have a billion devices that report their location to you at the same time they report their IP to you. That's basically Apple and Google. They have by far the best geolocation databases in the world, because they get constant updates of IP and location.

The trick is basically to make an app where people willingly give you their location, and then get a lot of people to use it. That's the best way to build an accurate geo-location database, and why every app in the world now asks for your location.

4-square had the right idea, they were just ahead of their time.

Re: How to build a IP geolocation database from scratch?

#90

Post by chaps »

reincoder wrote: post #88
chaps wrote: post #73

Earlier quoted context omitted.

And you're literally advertising this project as being helpful for targeted ads. So it's pretty clear from the get go that what you consider creepy isn't what I consider creepy. And having done enough reidentification work to scare myself, "thousands of people" might as well be a couple dozen or less. I get why you're defensive and why you think it's not creepy, but calling it a "meme" is insultingingly dismissive. J…

I grok and understand your concern. I am not being defensive; I am just trying to provide an explanation. I really enjoy having conversations like this with developers as honestly and empathetically possible. I apologize if I was rude in any way by saying the word "meme". I saw a sister comment and thought you were being sarcastic. There is a popular meme about "I have your IP address", so I thought you were referenc…

Thank you for the well thought out response. I disagree with just about everything you say, but I understand where you're coming from and I appreciate the validation that the use of a VPN is more important than it's ever been. As a professional courtesy: calling yourself "essential" is an enormous red flag and you might want to consider different phrasing.
Post reply on HN