Show HN: IP Geolocation and Threat Data API • Hacker News Board

Re: Show HN: IP Geolocation and Threat Data API

#51

Post by jonathan-kosgei » Wed, Apr 18, 2018, 11:39 PM UTC

jstarfish wrote: post #50

Looks neat, very clean. What factors go into determining whether an IP is a threat, and how often is this reviewed? One of the problems I have with most public intel (don't care if it's FireHOL, Crowdstrike, Alienvault or US-CERT) is that inevitably some GoDaddy (for example) site gets used by an APT and so a GoDaddy IP makes it onto a public blacklist, flagged as being abusive. But a billion other sites also share t…

Thanks!

You make a valid point. A number of VPN providers for example use GCE/AWS/Softlayer to host their services. And these IPs do get reassigned. One of the ways we mitigate this is by limiting the age of IP addresses in our lists to a maximum of 30 days. If an IP address hasn't been reported to have been responsible for malicious behavior for a period of 30 days it shouldn't be in our lists.

We're also mulling adding an is_cloud_provider field to the threat response object.

Re: Show HN: IP Geolocation and Threat Data API

#52

Post by taf2 » Wed, Apr 18, 2018, 11:53 PM UTC

jonathan-kosgei wrote: post #51

jstarfish wrote: post #50

Looks neat, very clean. What factors go into determining whether an IP is a threat, and how often is this reviewed? One of the problems I have with most public intel (don't care if it's FireHOL, Crowdstrike, Alienvault or US-CERT) is that inevitably some GoDaddy (for example) site gets used by an APT and so a GoDaddy IP makes it onto a public blacklist, flagged as being abusive. But a billion other sites also share t…

Thanks! You make a valid point. A number of VPN providers for example use GCE/AWS/Softlayer to host their services. And these IPs do get reassigned. One of the ways we mitigate this is by limiting the age of IP addresses in our lists to a maximum of 30 days. If an IP address hasn't been reported to have been responsible for malicious behavior for a period of 30 days it shouldn't be in our lists. We're also mulling ad…

is_cloud_provider, would be great!

Re: Show HN: IP Geolocation and Threat Data API

#53

Post by jonathan-kosgei » Wed, Apr 18, 2018, 11:54 PM UTC

taf2 wrote: post #52

jonathan-kosgei wrote: post #51

Earlier quoted context omitted.

Thanks! You make a valid point. A number of VPN providers for example use GCE/AWS/Softlayer to host their services. And these IPs do get reassigned. One of the ways we mitigate this is by limiting the age of IP addresses in our lists to a maximum of 30 days. If an IP address hasn't been reported to have been responsible for malicious behavior for a period of 30 days it shouldn't be in our lists. We're also mulling ad…

is_cloud_provider, would be great!

Thanks for the feedback!We'll probably push this by the weekend :)

Re: Show HN: IP Geolocation and Threat Data API

#54

Post by jedisct1 » Thu, Apr 19, 2018, 12:05 AM UTC

logronoide wrote: post #44

logronoide wrote: post #42

What tool do they use for https://status.ipdata.co ?

I will answer myself: https://updown.io I think I will use it in https://apility.io , a service that competes at some point with this.

https://apility.io/search/127.0.0.1

Not sure I would trust Apility.

Re: Show HN: IP Geolocation and Threat Data API

#55

Post by jimktrains2 » Thu, Apr 19, 2018, 4:07 AM UTC

jonathan-kosgei wrote: post #22

47 wrote: post #18

Do you provide local database? Making a web service call for every request seems like a performance bottle neck.

Hi, unfortunately we don't. However performance is very important to us which is why we have 11 endpoints around the world. And average ~65ms response times see status.ipdata.co.

65ms feels like a lot. I guess you can cache it so you're only calling once per IP.

Re: Show HN: IP Geolocation and Threat Data API

#56

Post by logronoide » Thu, Apr 19, 2018, 5:25 AM UTC

jedisct1 wrote: post #54

logronoide wrote: post #44

Earlier quoted context omitted.

I will answer myself: https://updown.io I think I will use it in https://apility.io , a service that competes at some point with this.

https://apility.io/search/127.0.0.1 Not sure I would trust Apility.

That means 127.0.0.1 is in these lists. We compile all different sources in one list you can customize. If you don’t trust one of several of them you can disable them using the api or the dashboard. There are over 100 now.

A cyber Intel engineer would perform a check in these lists to find out why this is a false positive. In this case there are bogons lists, and they are correct. Others, means that a popular malicious domain has changed the public ip to a private range. So the domain should be removed from them, something normally happens automatically in hours.

Re: Show HN: IP Geolocation and Threat Data API

#57

Post by zaarn » Thu, Apr 19, 2018, 6:34 AM UTC

One of the few services that geoip's my VPN correctly to germany. Too many of them pick it on France and I get shown lots of ads I don't understand. And no threat either (some blacklist me for sitting on an OVH network).

Good work!

Re: Show HN: IP Geolocation and Threat Data API

#58

Post by jonathan-kosgei » Thu, Apr 19, 2018, 9:55 AM UTC

zaarn wrote: post #57

One of the few services that geoip's my VPN correctly to germany. Too many of them pick it on France and I get shown lots of ads I don't understand. And no threat either (some blacklist me for sitting on an OVH network). Good work!

Thanks! :)

Re: Show HN: IP Geolocation and Threat Data API

#59

Post by jonathan-kosgei » Thu, Apr 19, 2018, 9:58 AM UTC

jimktrains2 wrote: post #55

jonathan-kosgei wrote: post #22

Earlier quoted context omitted.

Hi, unfortunately we don't. However performance is very important to us which is why we have 11 endpoints around the world. And average ~65ms response times see status.ipdata.co.

65ms feels like a lot. I guess you can cache it so you're only calling once per IP.

I understand what you mean, but every other provider from the tests I've done comes in at double our speeds some even over plain HTTP. We only serve requests over HTTPS

More importantly the performance is consistent and you'd get the same performance wherever you were in the world.

Re: Show HN: IP Geolocation and Threat Data API

#60

Post by jimktrains2 » Fri, Apr 20, 2018, 3:42 AM UTC

jonathan-kosgei wrote: post #59

jimktrains2 wrote: post #55

Earlier quoted context omitted.

65ms feels like a lot. I guess you can cache it so you're only calling once per IP.

I understand what you mean, but every other provider from the tests I've done comes in at double our speeds some even over plain HTTP. We only serve requests over HTTPS More importantly the performance is consistent and you'd get the same performance wherever you were in the world.

Why compare to another API endpoint? You should compare it against accessing a local database.