Live data from Hacker News

I made a simple geolocation service

maxkostinevich.com

41–50 of 187 posts

Re: I made a simple geolocation service

#41
Since most CDNs are already doing GeoIP lookups (for request headers or log entries), you can leverage that to provide the data back in the response body via origin, worker or even CDN edge config.

Programmatically populating the response body, as in the Cloudflare worker example from the post, is better than going to the origin just to echo some headers back in the response. To me, something like Fastly's VCL config language is even simpler. It directly executes on every CDN edge node worldwide upon request.

For example, I just whipped this up on Fastly using VCL. It returns GeoIP as json data for your IP at the root path:

http://geo.zombe.es

Or if you want a particular IP, just append it to the path:

http://geo.zombe.es/2a04:4e42:600::313

You could do the same via query params, headers, etc. Have URL endpoints that only return some of the data, and so forth.

The VCL syntax gets a little gross when you handle quoting strings and assembling json and testing if the string is empty, but it gets the job done.

Of course what you might want from GeoIP data may not be what you get. It's really kind of a useful kludge that gets treated sometimes as a panacea.

This dataset right now thinks that I'm about 5 miles east of my location, but when subnets are repurposed it could be much more significant. And the data sources are always changing, so who knows what it will think tomorrow.

Re: I made a simple geolocation service

#43
post #40

Side note: 6 million requests per month is only 2.2 requests per second, a raspberry PI could do this (technically).

That's only true if you assume a even distribution. If you cluster them the scenario is a bit different (your conclusion might still hold anyway)

Re: I made a simple geolocation service

#44
post #30

I've used the free MaxMinds database for a while but since the last year I've been using iplist.cc. It supports IPv4, IPv6, shows if an IP is tor, spam, which ASN and a lot more. It's also free and fast but with services like these I always wonder how long they manage to stay free.

Right. The first worry is "will this be here tomorrow?" The second is "how much more will they end up charging?"

I had good success with maxmind's free database. I can't find tooth results for accuracy but my sampling was pretty good. They also have an incentive to keep it.

Re: I made a simple geolocation service

#45
post #37

That country list doesn't have Kosovo (2008) or South Sudan (2011).

Kosovo is not universally recognised though, to be fair. But yes, I'd include it. Even if you are against in principle, it's de facto a state and treated as such even by many countries not recognising it in name.

Re: I made a simple geolocation service

#47
Can you make a geolocation service from static files? 2^32 IP4 addresses with 2 floats per address would take only 34 gigabytes of storage. Put 256 addresses in a given file, and turn the other three octets into folders. E.g. https://example.com/192/168/0.txt would contain the location for addresses 192.168.0.0-192.168.0.255.

Would this be cheaper than running these services?

Re: I made a simple geolocation service

#48
post #34

Also please see https://github.com/analogic/ipgeo daily actualized ip/country database with open license (shameless ad)

That would be a lot more appealing if it explained where the data is sourced from, included the update scripts, and how it is licensed - the only issue asking that question was closed without a word.

Re: I made a simple geolocation service

#49
post #41

Since most CDNs are already doing GeoIP lookups (for request headers or log entries), you can leverage that to provide the data back in the response body via origin, worker or even CDN edge config. Programmatically populating the response body, as in the Cloudflare worker example from the post, is better than going to the origin just to echo some headers back in the response. To me, something like Fastly's VCL config…

What does the code for this look like?

Re: I made a simple geolocation service

#50
post #40

Side note: 6 million requests per month is only 2.2 requests per second, a raspberry PI could do this (technically).

That's only true if you assume a even distribution. If you cluster them the scenario is a bit different (your conclusion might still hold anyway)

Even if all requests were clustered into one hour per day, it’s only 54 requests per second. Pretty sure a Pi would be up to the job.
Post reply on HN