Live data from Hacker News

Libpostal: A C library for parsing/normalizing street addresses around the world

github.com

1–10 of 26 posts

Re: Libpostal: A C library for parsing/normalizing street addresses around the world

#2
Already the name vs description reveals confusion: a street address and postal address does not have a 1:1 correlation even before taking postal codes/zip codes into account...

EDIT: examples includes differences in e.g visitor address vs. where mail delivery should happen; leaving out or adding details for one or the other (e.g many rural places you don't need to include road details for postal addresses).

Different people also address the same location differently. E.g I regularly have to tell delivery companies my address is in Surrey, even though my house has been in London for more than 50 years.

Re: Libpostal: A C library for parsing/normalizing street addresses around the world

#4
From the linked post

> Street addresses are among the more quirky artifacts of human language, yet they are crucial to the increasing number of applications involving maps and location.

The main goal seems to be positionning a point on a map.

As pointed out by the other comments, it’s fairly different from dealing with delivery addresses or legal addresses.

In particular it means parsing locations inside buildings (i.e. “3 appt of 2nd floor”, “Building 103 - code 17234, 34 foobar street”) with random info baked in for humans could easily trip it up and are not expected to either work properly.

Still looks like a pretty ambitious and interesting effort.

Re: Libpostal: A C library for parsing/normalizing street addresses around the world

#5
> Street addresses

Well, this already fails for places that don't address by street. You might think it's only some pre-industrial villages in the jungle, but examples would be some eastern European countries and Japan - some (but not all) buildings simply don't have a street address. Instead they have a number within a district. But sometimes it's a building number on a street, but it's distinct from the street's numbering system, so you can have Building 5 on st. Foo as a distinct address from 5th Foo st., where there is a completely different building. And of course there's no number on Foo st. that corresponds to Building 5. Another fun case is when there's a district Foo and a street Foo and e.g. Google Maps resolves "district Foo, building 5" as "No. 5, Foo st.". Or when the district has a number in it, so "district Foo 3, building 275" resolves to "district Foo, building 3", because of course the first Foo doesn't have a number in it - there's no Foo 1, only Foo, Foo 2, etc.

Generally all residential buildings built by the communist regime follow that system, while older buildings follow street numbers. Open Street Map actually deals amazingly well with our addresses, while Google Maps fails miserably most of the time. This is starting to become a problem as online services here are integrating google's mapping technology, e.g. an app for hailing taxis would ask you to type in your starting and destination address and if google can't make sense of it, the taxi can go to some completely wrong place. I can deal with it fine since I live here, but woe be any foreigner that would rely on Google Maps.

The postal system works just fine, but sometimes I have to enter a district name in the street field in online forms. As long as it arrives to the right country, the postal workers here can make sense of the address just fine.

All of these shenanigans do work in a hierarchical way, so you can pretty much expect to always have City, City sub-unit, Building designator as your address schema, but the actual category of City sub-unit and Building designator is sometimes "Street/number", sometimes "District/building number". You can of course simply ignore that and not have your system work in weird places, but if you're making a library for wide use and publishing it, I would appreciate it if you take into account that not everybody addresses by street/number.

Re: Libpostal: A C library for parsing/normalizing street addresses around the world

#6
post #4

From the linked post > Street addresses are among the more quirky artifacts of human language, yet they are crucial to the increasing number of applications involving maps and location. The main goal seems to be positionning a point on a map. As pointed out by the other comments, it’s fairly different from dealing with delivery addresses or legal addresses. In particular it means parsing locations inside buildings (i…

It explicitly is not a geocoder (which is address->location), it just parses and normalizes addresses.

It's meant to deal with something like "3 appt of 2nd floor", parsing and tagging "3 appt" as unit=apt. 3 and "of 2nd floor" as level=2, even if that string is mixed with further info like street and city and so on.

Re: Libpostal: A C library for parsing/normalizing street addresses around the world

#7
Nice work, although it is a bit slow: (avg 5 seconds per address parsing)

date && perl geo.pl && date Sat Dec 29 16:18:30 UTC 2018 country => united kingdom suburb => shoreditch house => the book club city => london postcode => ec2a 4rh road => leonard st house_number => 100-106 Sat Dec 29 16:18:35 UTC 2018

Re: Libpostal: A C library for parsing/normalizing street addresses around the world

#8
I work for UPS as software developer and surprisingly, I work for the department that is responsible for parsing addresses and matching them with actual physical addresses. We cover US, CA & EU (incl. UK). In our department, we have a guy whose entire career at UPS is nothing but maintaining library that parses addresses. It is very hard to get things right, unless you maintain that library all the time. Genuinely wish you good luck !

Re: Libpostal: A C library for parsing/normalizing street addresses around the world

#9
post #7

Nice work, although it is a bit slow: (avg 5 seconds per address parsing) date && perl geo.pl && date Sat Dec 29 16:18:30 UTC 2018 country => united kingdom suburb => shoreditch house => the book club city => london postcode => ec2a 4rh road => leonard st house_number => 100-106 Sat Dec 29 16:18:35 UTC 2018

It seems to have a whole lot of data that would probably need to be loaded. Maybe a lot of that time is spent in initialization. Is it faster at parsing a second, third, etc. address once loaded?

By the way, a more convenient way to benchmark Perl:

    perl -MBenchmark -e 'timethis(500, sub { ... your code here ... });'

Re: Libpostal: A C library for parsing/normalizing street addresses around the world

#10
post #4

From the linked post > Street addresses are among the more quirky artifacts of human language, yet they are crucial to the increasing number of applications involving maps and location. The main goal seems to be positionning a point on a map. As pointed out by the other comments, it’s fairly different from dealing with delivery addresses or legal addresses. In particular it means parsing locations inside buildings (i…

> The main goal seems to be positionning a point on a map.

It says "Actually geocoding addresses to a lat/lon" in the "Non-goals" section of README.md.

Post reply on HN