Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Libpostal: A C library for parsing/normalizing street addresses around the world

github.com

Post reply on HN
Print view View original

11–20 of 26 posts

Re: Libpostal: A C library for parsing/normalizing street addresses around the world

#11

Post by flexer2 »

libpostal is a pretty incredible open source project, but addresses are so complicated and nuanced that depending on what you’re doing, it might not be able to keep up. I work for a real estate tech company where we do a lot of address parsing and we had to move away from it because it’s just not quite powerful enough to handle all of the edge cases you find in US addresses. Right now we use SmartyStreets because their address parser is a bit better for our use case. Libpostal is a great general purpose library but depending on what level of accuracy you need, you might have to look for alternatives.

I spent time trying to use libpostal and build USPS address normalization rules on top of it but there are so many edge cases it was more cost effective to just purchase a solution from a vendor.

That is not to take away from this project — it’s quite good for a broad set of addresses across the world — but for narrow use cases such as ours it just couldn’t quite cut it.

Re: Libpostal: A C library for parsing/normalizing street addresses around the world

#12

Post by maxerickson »

newprint wrote: post #8

I work for UPS as software developer and surprisingly, I work for the department that is responsible for parsing addresses and matching them with actual physical addresses. We cover US, CA & EU (incl. UK). In our department, we have a guy whose entire career at UPS is nothing but maintaining library that parses addresses. It is very hard to get things right, unless you maintain that library all the time. Genuinely wi…

It was funded by the now defunct Mapzen. I have no idea, but I wouldn't be surprised if it isn't getting much work done anymore.

Re: Libpostal: A C library for parsing/normalizing street addresses around the world

#13

Post by eruci »

adrianmonk wrote: post #9
eruci wrote: post #7

Nice work, although it is a bit slow: (avg 5 seconds per address parsing) date && perl geo.pl && date Sat Dec 29 16:18:30 UTC 2018 country => united kingdom suburb => shoreditch house => the book club city => london postcode => ec2a 4rh road => leonard st house_number => 100-106 Sat Dec 29 16:18:35 UTC 2018

It seems to have a whole lot of data that would probably need to be loaded. Maybe a lot of that time is spent in initialization. Is it faster at parsing a second, third, etc. address once loaded? By the way, a more convenient way to benchmark Perl: perl -MBenchmark -e 'timethis(500, sub { ... your code here ... });'

You are correct. Only the first request takes about 4-5 seconds:

Start libpostal: Chong Co Thai Restaurant and Bar Shop 0039A Grand Central Shopping Centre 1-7 Dent St Toowoomba QLD 4350

house => chong co thai restaurant and bar shop 0039a grand central shopping centre

city => toowoomba

postcode => 4350

road => dent st

state => qld

house_number => 1-7

1: 4.10454607009888 seconds

Start libpostal: Little Plate Shop 9 11 Deodar Drive Burliegh Heads QLD 4220

house => little plate shop

city => heads

postcode => 4220

road => deodar drive burliegh

state => qld

house_number => 9 11

2: 0.000234127044677734 seconds

Start libpostal: Sheoak Shack Gallery Cafe 64 Fingal Rd Fingal Head NSW 2487

suburb => fingal head

house => sheoak shack gallery cafe

postcode => 2487

road => fingal rd

state => nsw

house_number => 64

3: 0.000188827514648438 seconds

Start libpostal: Chong Co Thai Restaurant and Bar Shop 0039A Grand Central Shopping Centre 1-7 Dent St Toowoomba QLD 4350

house => chong co thai restaurant and bar shop 0039a grand central shopping centre

city => toowoomba

postcode => 4350

road => dent st

state => qld

house_number => 1-7

4: 0.000257015228271484 seconds

Re: Libpostal: A C library for parsing/normalizing street addresses around the world

#14

Post by tyingq »

newprint wrote: post #8

I work for UPS as software developer and surprisingly, I work for the department that is responsible for parsing addresses and matching them with actual physical addresses. We cover US, CA & EU (incl. UK). In our department, we have a guy whose entire career at UPS is nothing but maintaining library that parses addresses. It is very hard to get things right, unless you maintain that library all the time. Genuinely wi…

Not your issue, but it makes me so angry that UPS charges me for address corrections ($12 a pop). The UPS supplied software often fails to identify missing suite number, can't figure out "State Route 123" vs "SR 123", etc. UPS is financially rewarded for bugs. Argh :)

Re: Libpostal: A C library for parsing/normalizing street addresses around the world

#15

Post by eruci »

newprint wrote: post #8

I work for UPS as software developer and surprisingly, I work for the department that is responsible for parsing addresses and matching them with actual physical addresses. We cover US, CA & EU (incl. UK). In our department, we have a guy whose entire career at UPS is nothing but maintaining library that parses addresses. It is very hard to get things right, unless you maintain that library all the time. Genuinely wi…

BTW, I'm a guy who has been building address parsing software since 2005. ( geocoder.ca, geocode.xyz ) It is HARD.

Currently I'm using machine learning similarly to libpostal to improve my software.

It works better than libpostal in some cases (for eg:

USA: Start libpostal: 751 FAiR OKS AVENUE PASADNA CA

road => fair oks avenue pasadna

state => ca

house_number => 751

Geocoder.ca 751 FAiR OKS AVENUE PASADNA CA

https://geocoder.ca/?locate=751+FAiR+OKS+AVENUE+++PASADNA+CA...

stnumber: 751

staddress: N Fair Oaks Ave

city: Pasadena

prov: CA

postal: 91103-3069

libpostal from Little Plate Shop 9 11 Deodar Drive Burliegh Heads QLD 4220

house => LITTLE PLATE SHOP

city => HEADS

postcode => 4220

road => DEODAR DRIVE BURLIEGH

state => QLD

house_number => 9 11

Geocode.xyz Little Plate Shop 9 11 Deodar Drive Burliegh Heads QLD 4220

addresst: DEODAR DR

region: QLD

postal: 4220

stnumber: 11

prov: AU

city: BURLEIGH HEADS

countryname: Australia

confidence: 0.7

And it works worse in some other cases...

Anyways, good luck to this developer. I don't think anyone will ever produce a solution that works better than all others, but it is better if more of us try.

Re: Libpostal: A C library for parsing/normalizing street addresses around the world

#16

Post by rfw »

Asooka wrote: post #5

> Street addresses Well, this already fails for places that don't address by street. You might think it's only some pre-industrial villages in the jungle, but examples would be some eastern European countries and Japan - some (but not all) buildings simply don't have a street address. Instead they have a number within a district. But sometimes it's a building number on a street, but it's distinct from the street's nu…

If you watch the GIF, the library parses Japanese addresses just fine.

Re: Libpostal: A C library for parsing/normalizing street addresses around the world

#18

Post by taneliv »

Asooka wrote: post #5

> Street addresses Well, this already fails for places that don't address by street. You might think it's only some pre-industrial villages in the jungle, but examples would be some eastern European countries and Japan - some (but not all) buildings simply don't have a street address. Instead they have a number within a district. But sometimes it's a building number on a street, but it's distinct from the street's nu…

It worked quite fine for fairly approximate things like "Tetuán, Madrid, España" (district, city, country). However, it seemed to exhibit a tendency to attribute suburbs and districts as houses, at least with the handful of Madrid addresses I happened to have at hand.

If you have a use case for it and data to match, why not give it a try?

Post reply on HN