Live data from Hacker News

In Defense of OpenStreetMap's Data Model

stevecoast.substack.com

11–20 of 130 posts

Re: In Defense of OpenStreetMap's Data Model

#11
I’ve started playing with data from OpenStreetMap. It started with me trying to fetch all the places where I could get water when moving around Copenhagen, which turned out not to be as easy as first envisioned, because OSM seems to have a lot of different ways to categorise available water, which makes sense, OSM and the tagging system isn't there to support only my usecase, and describing my idea doesn't fit 1:1 with the model.

I identified the following tags to look out for:

amenity=drinking_water, https://wiki.openstreetmap.org/wiki/Tag:amenity%3Ddrinking_w...

man_made=water_tap, https://wiki.openstreetmap.org/wiki/Tag:man_made%3Dwater_tap

amenity=water_point, https://wiki.openstreetmap.org/wiki/Tag:amenity%3Dwater_poin...

drinking_water=*, https://wiki.openstreetmap.org/wiki/Key:drinking_water

It's a tough problem to map out the world and describe it, especially when everyone can add or modify the data, but anything that could improve the experience of importing like osm2pgsql would be welcome.

Re: In Defense of OpenStreetMap's Data Model

#13
In the first part of the article I was thinking, oh, maybe Steve Coast isn't such a jerk after all.

Then I got to the meat of it. Oh dear.

As one of the many many people who has had to deal with OSM data, I curse people with this attitude that the mess is somehow desirable or necessary. It's not. There is a long spectrum between totally free form and completely constrained, and OSM's data model is painfully down the wrong end, and causes enormous harm to all kinds of potential reuses of the data.

It also causes harm to the people creating data. Try adding bike paths and figuring out what tags are appropriate in your area. Try working out how to tag different kinds of parks, or which sorts of administrative boundaries should be added or how they should be maintained. It puts many people off, me included.

Bah.

Re: In Defense of OpenStreetMap's Data Model

#14

What is stopping users who have a problem with the model from transforming the data into a form that is better for their use case?

It is surprisingly difficult to say which closed ways are areas and which are not. This depends entirely on tags of the way and is only solved by heuristics.

https://github.com/tyrasd/osm-polygon-features

Re: In Defense of OpenStreetMap's Data Model

#15
I know it’s nothing to do with the main thrust of the article, but the author fundamentally misrepresents KYC. Know-your-customer is a facet of anti-money laundering and anti-corruption regulation. It has nothing to do with talking to users.

Re: In Defense of OpenStreetMap's Data Model

#16
The exact same argument that praises OSM's super flexible tagged node data model should also praise MS Excel for the number of things that can be achieved in the world of business with just a grid of boxes.

Both have been hugely successful, and both have the same pile of downsides.

Re: In Defense of OpenStreetMap's Data Model

#17
post #11

I’ve started playing with data from OpenStreetMap. It started with me trying to fetch all the places where I could get water when moving around Copenhagen, which turned out not to be as easy as first envisioned, because OSM seems to have a lot of different ways to categorise available water, which makes sense, OSM and the tagging system isn't there to support only my usecase, and describing my idea doesn't fit 1:1 wi…

I don't understand how this doesn't fit your use case. The tags are for different things, e.g.

> for places where you can get larger amounts of "drinking water" for filling a fresh water holding tank, such as found on caravans, RVs and boats

versus

> a man-made construction providing access to water, supplied by centralized water distribution system (unlike in case of man_made=water_well [...]). The tag man_made=water_tap is used for publicly usable water taps, such as those in the cities and graveyards. Water taps may provide potable and technical water, which can be specified with drinking_water=yes and drinking_water=no.

And another tag for when you're not mapping a separate water point, but indicating whether a given feature has drinking water (for example a well or mountain hut).

You're saying that it's tough when anyone can mess with the data rather than working in a structured way, but these tags have distinct definitions and seem perfectly sensible to me (there are much worse examples like highway=track, which spawned huge discussions in various places within the community). How do these tags not match your use case to select which tags you need and display those in the way you want (e.g. as list or map)?

Re: In Defense of OpenStreetMap's Data Model

#18

The proposed improvements would obsolete a bunch of problems such as broken polygons [1] which happen regularly. They would also make processing OSM more accessible without needing to randomly seek over GBs of node locations just to assemble geometries which takes a significant runtime percentage of osm2pgsql. For me Steve Coast lost his credibility when he joined the closed and proprietary what3words. [1] https://wi…

But you don't need to complicate the storage format to fix a problem like that. You can build validation tools that will check whether the stored data conforms to the correct specified geometry, and only emit valid polygons to later tools in the pipeline when they do. "Be liberal in what you accept and strict in what you send" is still a good principle. The problem with rejecting invalid structures at the data storag…

The best thing is not to allow invalid geometries to begin with. Any validation would need to be done in an off-line fashion for a number of reasons (such as needing to retrieve any referred OSM elements), and by that time you can't automatically revert offending changes as any revert carries a chance of an object version conflict.

Re: In Defense of OpenStreetMap's Data Model

#19

The proposed improvements would obsolete a bunch of problems such as broken polygons [1] which happen regularly. They would also make processing OSM more accessible without needing to randomly seek over GBs of node locations just to assemble geometries which takes a significant runtime percentage of osm2pgsql. For me Steve Coast lost his credibility when he joined the closed and proprietary what3words. [1] https://wi…

But you don't need to complicate the storage format to fix a problem like that. You can build validation tools that will check whether the stored data conforms to the correct specified geometry, and only emit valid polygons to later tools in the pipeline when they do. "Be liberal in what you accept and strict in what you send" is still a good principle. The problem with rejecting invalid structures at the data storag…

The "expression" layer of the data model has had 20 years to evolve and has largely been static for a decade.

Making everything slower and harder to retain flexibility you don't need isn't a great tradeoff.

Post reply on HN