Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A fast, offline reverse geocoder in Python

github.com

Post reply on HN
Print view View original

31–40 of 46 posts

Re: A fast, offline reverse geocoder in Python

#31

Post by Animats »

While we're on this subject, is there a good, free street address parser that will work for at least the US, Canada, UK, and the major EU countries? I've tried most of the available ones, and they can parse about 90-95% of business addresses.

(Regular expressions don't work well for this. Neither does starting from the beginning of the address. Proper address parsing starts at the end of the address and works backwards, with the information found near the end, such as country name and postal code, used to disambiguate the information found earlier.)

Re: A fast, offline reverse geocoder in Python

#32

Post by shawn-butler »

bzz01 wrote: post #27

You can't really use KD trees with lat/lon coordinates, at least you can't use euclidean distance there for nearest neighbor search. First, longitude wraps from -180 to +180 at antimeridian, meaning distance calculations will fail there; second, and I'd say more importantly, one degree longitude length in meters differs a lot depending on latitude; meaning this library will be heavily biased towards longitudal neighb…

I can recommend the excellent Geographic lib::Geodesic for this having used it in the past.

http://geographiclib.sourceforge.net

There is a python implementation available as well. http://pypi.python.org/pypi/geographiclib

Re: A fast, offline reverse geocoder in Python

#33

Post by Stratoscope »

yellowbkpk wrote: post #21
thampiman wrote: post #19

Earlier quoted context omitted.

Very cool project. Out of curiosity, how do you test the validity of the addresses?

All of the addresses come from "authoritative" datasets (i.e. from local or state governments) so we're assuming them to be correct.

Interesting project!

I assume you don't have a manual procedure to report and correct individual addresses that are in error, and the trick would be to contact the agency that provided the erroneous dataset, is that right?

It would be useful to have a way to find out "where did the data for this location come from, and who do I contact to correct it?"

For example, our house doesn't have an outline on your sample map, although there is a red dot there. Our street address shows up on the house next door, and the neighboring houses in that direction are shifted similarly.

Most other online maps for our location have the same error or similar. I think the confusion happened because we are on a corner and some years ago a previous owner changed our street address from one street to the other. So whatever agency maintains this data goofed something up in the process.

If there were an easy way to find out who this mystery agency is, then I could help them sort out this mixup. :-)

Re: A fast, offline reverse geocoder in Python

#34

Post by maxerickson »

Stratoscope wrote: post #33
yellowbkpk wrote: post #21

Earlier quoted context omitted.

All of the addresses come from "authoritative" datasets (i.e. from local or state governments) so we're assuming them to be correct.

Interesting project! I assume you don't have a manual procedure to report and correct individual addresses that are in error, and the trick would be to contact the agency that provided the erroneous dataset, is that right? It would be useful to have a way to find out "where did the data for this location come from, and who do I contact to correct it?" For example, our house doesn't have an outline on your sample map,…

It isn't really user friendly, but I would say it is fairly easy to find what information they are using by looking at the source files:

https://github.com/openaddresses/openaddresses/tree/master/s...

(In the US the data might come from someone who accepts updates from someone else, but it is usually the state, county or city

Re: A fast, offline reverse geocoder in Python

#35

Post by unclesaamm »

Animats wrote: post #31

While we're on this subject, is there a good, free street address parser that will work for at least the US, Canada, UK, and the major EU countries? I've tried most of the available ones, and they can parse about 90-95% of business addresses. (Regular expressions don't work well for this. Neither does starting from the beginning of the address. Proper address parsing starts at the end of the address and works backwar…

The most principled approach I've seen on this is at https://github.com/datamade/usaddress. They use tagged training data and conditional random fields. I haven't seen comparisons with other systems, but it's worked well enough for my projects.

Though as the name suggests, it's only trained for US addresses.

Re: A fast, offline reverse geocoder in Python

#37

Post by tomcam »

mvexel wrote: post #25
yellowbkpk wrote: post #18

If you're interested in making both forward and reverse geocoding better, please consider paying attention to a project I started and help maintain called OpenAddresses: http://openaddresses.io The goal is to collect address datasets so that forward and reverse geocoding is an easier problem to solve. A contributor wrote an excellent overview of the project the other day: https://medium.com/colemanm/creating-an-open-…

It's a nice overview but it glosses over the fact that while the OA database is governed by a CC0 license, the individual address collections in the database are still governed by their own licenses which can be (much) more restrictive. The fact that you can download the data doesn't mean you can use it the way you want. The OA web site hints at this but doesn't address (ah-hah) that underlying problem. That doesn't…

Lists laid out in an obvious format are not subject to copyright.

Re: A fast, offline reverse geocoder in Python

#38

Post by belorn »

thampiman wrote: post #24
natch wrote: post #23

Kudos for a very well done README (and it's not just cribbed from the original project, it explains the new stuff very well and tells what the project is, and gives credit back). So many projects neglect the README. One question - is it OK to put an MIT license on something that is based on LGPL code? I don't know enough about how the LGPL works (I do know it is less "infective" than plain GPL). Well two questions: p…

Thanks for that comment! Good question regarding the license. I'm not too sure about that. I'd appreciate it if someone could shed some light on it. Regarding the version, I've only tested it on python2. I should add that in the README. Thanks!

If its based on LGPL code, then any original code need to retain it's license.

Beyond that, everything regarding licenses look good if you ask me. MIT is both compatible if you modify the LGPL code or if you link with it, so one can use either of the two licenses.

Re: A fast, offline reverse geocoder in Python

#39

Post by rbp »

thampiman wrote: post #24
natch wrote: post #23

Kudos for a very well done README (and it's not just cribbed from the original project, it explains the new stuff very well and tells what the project is, and gives credit back). So many projects neglect the README. One question - is it OK to put an MIT license on something that is based on LGPL code? I don't know enough about how the LGPL works (I do know it is less "infective" than plain GPL). Well two questions: p…

Thanks for that comment! Good question regarding the license. I'm not too sure about that. I'd appreciate it if someone could shed some light on it. Regarding the version, I've only tested it on python2. I should add that in the README. Thanks!

hi, I am the author of the original library, which uses the LGPL license. My reason for using LGPL was so people would be obligated to share their modifications, so I would expect this is not compatible with MIT.

Re: A fast, offline reverse geocoder in Python

#40

Post by gabemart »

mvexel wrote: post #25
yellowbkpk wrote: post #18

If you're interested in making both forward and reverse geocoding better, please consider paying attention to a project I started and help maintain called OpenAddresses: http://openaddresses.io The goal is to collect address datasets so that forward and reverse geocoding is an easier problem to solve. A contributor wrote an excellent overview of the project the other day: https://medium.com/colemanm/creating-an-open-…

It's a nice overview but it glosses over the fact that while the OA database is governed by a CC0 license, the individual address collections in the database are still governed by their own licenses which can be (much) more restrictive. The fact that you can download the data doesn't mean you can use it the way you want. The OA web site hints at this but doesn't address (ah-hah) that underlying problem. That doesn't…

As the (dead) sibling comment points out, my (non-lawyer) understanding of US copyright law is that simple collections of facts, when compiled in a way that requires no creativity, do not enjoy any copyright protection at all.

I would be surprised if a simple list of addresses, even a very large one, is something that could be subject to copyright.

Post reply on HN