Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenStreetMap is the Most Important thing in Geo

gisdoctor.com

Post reply on HN
Print view View original

111–120 of 141 posts

Re: OpenStreetMap is the Most Important thing in Geo

#111

Post by skimpycompiler »

jpatokal wrote: post #104
skimpycompiler wrote: post #92

Earlier quoted context omitted.

It's a mobile app primarily. OSRM is a nice project - out of the box. The lead went to some giant - so project is currently a bit stray but there are also some other alternatives on github which work well but might not be as mature as OSRM. Google caches insane amounts of things. What one might think is insane algorithms, it's probably just Googles multimillion cluster memory. Google is also lucky that no one needs t…

Assume there are a billion places on Earth. (As there are ~7B people, each of whom lives in a "place", this is already an underestimate.) If it takes a single byte to store the distance between each of these places, you would need an exabyte of storage just for this single byte.

You are really arguing that?

What system would calculate a shortest path between Sydney and London? There's an incredible amount of sparseness in the matrix.

OSRM builts a hierarchy that allows fast computation of shortest paths for the whole world map. It takes about 4 hours on my server, fills around 40GB of RAM, and then after that I can magically do insane amounts of shortest path queries.

Paper below has an even faster algorithm that has an operation of finding shortest paths equivalent to just several reads in memory (according to experimental results in the paper, it's roughly equal to 5 reads, meaning it's just 5 times slower than that 1E18 table we'd have). It works on a single workstation.

It might not be a pairwise cache but man, this is some advanced stuff and I'm sure Google's engineers wouldn't think of having 1E18 elements matrix.

That's why I'm a bit surprised by the pricing and the limits. I guess network traffic bandwidth costs.

http://research.microsoft.com/apps/pubs/default.aspx?id=1456...

Re: OpenStreetMap is the Most Important thing in Geo

#112

Post by voltagex_ »

Xylakant wrote: post #27
Brakenshire wrote: post #21

Earlier quoted context omitted.

Similarly, a lot of charitable organizations that could be releasing data to build on top of the OSM dataset and the tooling that exists, instead create their own data silos. For instance, Sustrans (the cycling charity in the UK), and the Ramblers (the walking/hiking charity in the UK). I get the impression that managers are coming up with ideas for their digital projects without any reference to the technical landsc…

http://wheelmap.org/ , a map for wheelchair accessible places builds on OSM.

Just a warning for any other contributors, at the moment the changeset that WheelMap posts to OSM puts a huge bounding box on the map which makes it really hard for other OSM users to see what's been changed.

https://github.com/sozialhelden/wheelmap/issues/29

Re: OpenStreetMap is the Most Important thing in Geo

#113

Post by mikekchar »

maxerickson wrote: post #107
mikekchar wrote: post #106

Earlier quoted context omitted.

That is amazing! One thing I'm very interested in is the vertical data which Strava seems to be very good at interpolating from GPS traces. Do you know if they contribute that data anywhere?

They pull the elevation data from government data sets: https://strava.zendesk.com/entries/20965883-Elevation-for-Yo...

Thank you! That was very interesting. I had assumed that they were interpolating GPS data, but I guess not. It seems they are smoothing it, though.

That being the case, perhaps I should write the program I was thinking of writing. Even though GPS elevation data is really, truly horrible I think you could probably use statistical techniques to get good data with enough GPS traces. Even routes that are mildly popular can have thousands, or tens of thousands of rides per year. Each one of those rides on Strava has GPX data associated with it. I'm sure it would be more than enough.

Unfortunately, I have about a million other projects on the go :-P

Re: OpenStreetMap is the Most Important thing in Geo

#114

Post by aidenn0 »

icewater0 wrote: post #75
lallysingh wrote: post #2

It is awesome. Here's a docker image that'll serve as a tile server. Add a JS library (I use d3) and a OSM snapshot and go. https://github.com/geo-data/openstreetmap-tiles-docker Note: a bit finicky.

I was going to post a comment how I would like to have a local instance of OSM for my state/region. Is it possible to calculate directions/routing locally too?

try out maps.me

Re: OpenStreetMap is the Most Important thing in Geo

#115

Post by voltagex_ »

AReallyGoodName wrote: post #94
skimpycompiler wrote: post #55

Earlier quoted context omitted.

Google Maps is just too expensive and its pricing model isn't exponential at all, it's not even quadratic. Very common operation is looking for pairwise distance matrix. Pricing and limits are totally irrational. 100,000 daily limit is totally impossible for an app with 10 users. 100x100 = 10,000 :D Given that Google probably has all of the distances cached somewhere on their servers, I see no technical reason why th…

Online geocoding service are always the worst way to go. You only use the online services if you don't have an alternative dataset. The osm datasets actually aren't that big, the entire PBF formatted dataset is Also it's just completely fucking ridiculous to send a request over the internet to perform a calculation. I encountered the issue myself ages ago where i had a task to reverse geocode for a railways GPS syste…

What was your dataset for your reverse geocodes? The GeoNames dataset can't be that accurate, can it?

Re: OpenStreetMap is the Most Important thing in Geo

#117

Post by lucb1e »

Doctor_Fegg wrote: post #33
Brakenshire wrote: post #21

Earlier quoted context omitted.

Similarly, a lot of charitable organizations that could be releasing data to build on top of the OSM dataset and the tooling that exists, instead create their own data silos. For instance, Sustrans (the cycling charity in the UK), and the Ramblers (the walking/hiking charity in the UK). I get the impression that managers are coming up with ideas for their digital projects without any reference to the technical landsc…

For two UK charities I've worked with, the reason is that they've been tempted by the siren voices of Google. Google offer a tempting deal to charities: free mapping, and the chance to get your content hosted on Google Maps. So, for example, Sustrans' cycle routes are now integrated into the Google's UK mapping data. This seems to appeal particularly to the higher-ups. The developers/GIS team may be more au fait with…

I have a similar experience regarding choosing OSM over people's go to mapping service (which looks completely free to them because Google doesn't ask them to pay). One way to deal with it is just not taking about the decision when they don't even know all the options. Just make it work and they'll probably be fine with it.

Re: OpenStreetMap is the Most Important thing in Geo

#118

Post by mkhpalm »

ancymon wrote: post #82
Doctor_Fegg wrote: post #65

Earlier quoted context omitted.

There are several geocoders available for OSM. To say that "the database is inconsistent [and] contains many errors", and then back that up solely with a criticism of one piece of client software, makes no sense.

I've backed the statement that OSM contains errors by saying that there are duplicated addresses. I've backed the statement that OSM is inconsistent by saying that sometimes streets in OSM contain full name and sometime only most important part (and that makes Nominatim unable to search for addresses).

I agree, we started looking at OSM's address data sets only to find them pretty sparse. For the areas we investigated I'd estimate around 5% coverage from community efforts. Made me wonder why nobody's done something stupid like grab the files from http://openaddresses.io and create a reasonable starting point? But that got me thinking why even bother with Nominatim vs just throwing all that data into elastic for better fuzzy searches or match.

Re: OpenStreetMap is the Most Important thing in Geo

#119

Post by jarek »

voltagex_ wrote: post #112
Xylakant wrote: post #27

Earlier quoted context omitted.

http://wheelmap.org/ , a map for wheelchair accessible places builds on OSM.

Just a warning for any other contributors, at the moment the changeset that WheelMap posts to OSM puts a huge bounding box on the map which makes it really hard for other OSM users to see what's been changed. https://github.com/sozialhelden/wheelmap/issues/29

In practice, though, a lot of other editors also make huge bounding-box edits, to the point where I've found box history basically useless in London even ignoring wheelmap edits. And I'd take having accessibility data over cleaner edit history.

Re: OpenStreetMap is the Most Important thing in Geo

#120

Post by adrianN »

jpatokal wrote: post #104
skimpycompiler wrote: post #92

Earlier quoted context omitted.

It's a mobile app primarily. OSRM is a nice project - out of the box. The lead went to some giant - so project is currently a bit stray but there are also some other alternatives on github which work well but might not be as mature as OSRM. Google caches insane amounts of things. What one might think is insane algorithms, it's probably just Googles multimillion cluster memory. Google is also lucky that no one needs t…

Assume there are a billion places on Earth. (As there are ~7B people, each of whom lives in a "place", this is already an underestimate.) If it takes a single byte to store the distance between each of these places, you would need an exabyte of storage just for this single byte.

Nobody stores all shortest paths, that's just silly. With some rather simple preprocessing you can answer shortest paths queries on the US road network in microseconds [1], with preprocessing time of less than a day on a single machine. Surely Google can scale that up to larger graphs.

[1]http://people.mpi-inf.mpg.de/~dmatijev/papers/DIMACS06.pdf

Post reply on HN