Map Vectorizer – Map polygon and feature extractor
1–10 of 22 posts
Re: Map Vectorizer – Map polygon and feature extractor
#2Very very cool. Anybody know of similar projects?
I think I'll give it a whirl on some old 60's fenceline maps I have around.
Re: Map Vectorizer – Map polygon and feature extractor
#3What a cool project. Kudos to the NYPL for releasing this.
I've been working on something similar recently using OpenCV (though haven't done much on it yet). My use case is to find paths through electrical schematics. I figured my problem was close to map vectorization so I searched around for an existing library in that space but didn't have any luck.
Will be interesting to see how they've approached it and what sort of results I could get using their library.
Re: Map Vectorizer – Map polygon and feature extractor
#4I've always wanted the equivalent of * Maps with a time dimension. The data is constantly changing, but imagine if you could push the slider back hundreds of years and see how a city evolved. There is a whole "dark" dimension of data out there that is only captured in print. (e.g. NYC business directories from the 19th/20th century -- being able to see what a particular address used to be) Adding historical map data is the base layer for capturing this data from print.
Re: Map Vectorizer – Map polygon and feature extractor
#5This is a great idea but I question the process in the example - should they not be capturing the lines and then dividing into areas rather than trying to draw areas instantly (the areas being inside the line with random width spaces between)? Also - a good process to capture the information will probably vary significantly from map to map.
Re: Map Vectorizer – Map polygon and feature extractor
#6This is pretty cool. A lot of manual work on OSM, back when I was with some OSM nerds in DC, faced the problem of manual imports and a tool like this would have been very, very useful.
Thank you NYPL!
Re: Map Vectorizer – Map polygon and feature extractor
#7I've always wanted the equivalent of * Maps with a time dimension. The data is constantly changing, but imagine if you could push the slider back hundreds of years and see how a city evolved. There is a whole "dark" dimension of data out there that is only captured in print. (e.g. NYC business directories from the 19th/20th century -- being able to see what a particular address used to be) Adding historical map data…
I'm part of the team at NYPL Labs that's been working on these historical geospatial projects for the past few years (The Vectorizer is the work of our own @MGA), and that's EXACTLY what we've been working toward. For almost 4 years, we've had staff and volunteers going over scans of geo-rectified (stitching and stretching a raster image so it aligns with geospatial coordinates) historical insurance maps of NYC meticulously extracting the information on there. Namely we go after the outlines of buildings to capture the amazingly detailed datapoints these maps had about every building in the city all the way back to the first half of the 19th century.
These are nice to have for researchers, but the real purpose of collecting this is, just as you note, to unlock the hidden historical geospatial data in textual materials. Once we've got all those names of places, their addresses, their lat/lon coordinates, and their timeframes of existence, we can start to search through texts to find linkages. Old city directories (they're basically books of ghosts) start to show you who lived and worked where [1] (and in the process starts to get you more names you can associate with these places), address matches in historical newspapers start to show you what happened in these places, and the maps start to become this geospatial backbone to traverse across tons of different datasets.
The Vectorizer is so freaking cool for so many reasons, but mostly because it's going to let us actually get through these insurance atlases to collect this data before we all die (one of our favorites is the 1854 William Perris Atlas [2][3] but it took nearly 3 years to actually get through the 64,000+ buildings in Manhattan south of 42nd st) so we can start doing this kind of querying with it. The real geniuses behind all this, our Geospatial Librarian Matt Knutzen and the team at Topomancy, have been working on an experimental gazetteer [4] so that we'll finally have this as a public web service for people to hack on all these places as we collect and conflate them. Give us a few months...
In the meantime, sign up for the Open Historical Maps project listserv [5] that some of the OSM crew is working on (including the geniuses at Topomancy).
Also, this came out of a historical geospatial hack day [5] we threw a few months back, which you should check out if you want to play around with some of our data sources for this kind of work or for building something else out of historical NYC's geospatial footprint.
[1]: http://andrewxhill.github.io/cartodb-examples/scroll-story/b... [2]: http://maps.nypl.org/warper/layers/861 Tileserver, please forgive me for linking to you [3]: http://aaronland.info/nypl-perris/ YEAH SHAPEFILES! [4]: http://vimeopro.com/openstreetmapus/state-of-the-map-us-2013... Schuyler Earle's presentation on their version of historical gazetteer they're building for the Library of Congress at State of The Map US 2013 [5]: http://www.nypl.org/blog/2013/07/12/maphack-hacking-nycs-pas...
Re: Map Vectorizer – Map polygon and feature extractor
#8This is pretty cool. A lot of manual work on OSM, back when I was with some OSM nerds in DC, faced the problem of manual imports and a tool like this would have been very, very useful. Thank you NYPL!
OSM's been pretty hardcore about their no import policy because of accuracy issues in validation. They want to map these places themselves and don't want to trust others for accuracy, let alone the legal issues that come with importing data (ugh).
We, however are far more lenient (mostly because we can't afford to build a time machine to map the past ourselves).
On the other hand, Mike Migurski's Green Means Go [1] project is fantastic for figuring out where batch imports into OSM will be greeted with confetti and parades for filling out parts of the US without enough coverage to warrant anti-import protectionism.
Re: Map Vectorizer – Map polygon and feature extractor
#9This is a great idea but I question the process in the example - should they not be capturing the lines and then dividing into areas rather than trying to draw areas instantly (the areas being inside the line with random width spaces between)? Also - a good process to capture the information will probably vary significantly from map to map.
It does vary from map to map although it is optimized for insurance maps (or any map made mostly with clearly delineated polygons). We've included a config file that you would modify to suit the maps you are working with. The current process takes advantage of preexisting features in mapping tools and adds feature detection (e.g. polygon is/isn't a building) and concave hulls.
We do welcome input and merge requests to improve the tool!
Re: Map Vectorizer – Map polygon and feature extractor
#10What a cool project. Kudos to the NYPL for releasing this. I've been working on something similar recently using OpenCV (though haven't done much on it yet). My use case is to find paths through electrical schematics. I figured my problem was close to map vectorization so I searched around for an existing library in that space but didn't have any luck. Will be interesting to see how they've approached it and what sor…
OpenCV is used here only for the "has dot/cross" aspect of feature detection (and it is very primitive still). The polygons themselves are more a work of R and GDAL.