In Defense of OpenStreetMap's Data Model
stevecoast.substack.com
In Defense of OpenStreetMap's Data Model
1–10 of 130 posts
Re: In Defense of OpenStreetMap's Data Model
#2[deleted]
Re: In Defense of OpenStreetMap's Data Model
#3Skip about 1/3 of the way down and the OSM article starts. “This is why bad design is everywhere...”
Re: In Defense of OpenStreetMap's Data Model
#4Skip about 1/3 of the way down and the OSM article starts. “This is why bad design is everywhere...”
Seriously, there's a good article buried in there. Are we going to see the recipe site SEO anecdotes propagate.
Re: In Defense of OpenStreetMap's Data Model
#5Skip about 1/3 of the way down and the OSM article starts. “This is why bad design is everywhere...”
The entire article can be summed up as: “OSM stores maps as graphs, in flat files where each line is either a node, an ordered list of nodes, or metadata. The graph nodes can be arbitrarily ordered in OSM files, which leads to computational complexity when parsing them. This is not a bad thing, since it means that the spec for OSM files can be extremely simple, which makes it easy for people to contribute to OSM. Other mapping formats optimized for parsing speed require a lot of irrelevant fluff that makes them much harder to understand by human contributors.”
Ironically, 95% of this article is irrelevant fluff that does not make it any easier for the reader to understand.
Re: In Defense of OpenStreetMap's Data Model
#6I think those are the important bits:
> The Engineering Working Group (EWG) of the OSMF has “commissioned” (I think that’s OSMF language for paid) a longstanding proponent of rules and complexity to, uh, investigate how to add rules and complexity to OSM.
> [...]
> Let us pray that the EWG is just throwing Jochen a bone to go play in the corner and stop annoying the grownups.
It's a "response" to https://blog.openstreetmap.org/2022/06/02/announcement-data-...
Re: In Defense of OpenStreetMap's Data Model
#7Skip about 1/3 of the way down and the OSM article starts. “This is why bad design is everywhere...”
The entire article can be summed up as: “OSM stores maps as graphs, in flat files where each line is either a node, an ordered list of nodes, or metadata. The graph nodes can be arbitrarily ordered in OSM files, which leads to computational complexity when parsing them. This is not a bad thing, since it means that the spec for OSM files can be extremely simple, which makes it easy for people to contribute to OSM. Oth…
> OSM stores maps as graphs, in flat files where each line is either a node, an ordered list of nodes, or metadata. The graph nodes can be arbitrarily ordered in OSM files, which leads to computational complexity when parsing them. This is not a bad thing, since it means that the spec for OSM files can be extremely simple, which makes it easy for people to contribute to OSM.
That's actually a sensible design. Treat user-facing stored data as user interface. If you need efficient processing of that data, such as fast parsing, you can always build it elsewhere, such as by caching that data into an intermediate structure that is recompiled whenever the user data changes.
Re: In Defense of OpenStreetMap's Data Model
#8The proposed improvements would obsolete a bunch of problems such as broken polygons [1] which happen regularly. They would also make processing OSM more accessible without needing to randomly seek over GBs of node locations just to assemble geometries which takes a significant runtime percentage of osm2pgsql.
For me Steve Coast lost his credibility when he joined the closed and proprietary what3words.
[1] https://wiki.openstreetmap.org/wiki/OSM_Inspector/Views/Mult...
Re: In Defense of OpenStreetMap's Data Model
#9Earlier quoted context omitted.
The entire article can be summed up as: “OSM stores maps as graphs, in flat files where each line is either a node, an ordered list of nodes, or metadata. The graph nodes can be arbitrarily ordered in OSM files, which leads to computational complexity when parsing them. This is not a bad thing, since it means that the spec for OSM files can be extremely simple, which makes it easy for people to contribute to OSM. Oth…
> OSM stores maps as graphs, in flat files where each line is either a node, an ordered list of nodes, or metadata. The graph nodes can be arbitrarily ordered in OSM files, which leads to computational complexity when parsing them. This is not a bad thing, since it means that the spec for OSM files can be extremely simple, which makes it easy for people to contribute to OSM. That's actually a sensible design. Treat u…
Someone gets it :-)
Re: In Defense of OpenStreetMap's Data Model
#10The proposed improvements would obsolete a bunch of problems such as broken polygons [1] which happen regularly. They would also make processing OSM more accessible without needing to randomly seek over GBs of node locations just to assemble geometries which takes a significant runtime percentage of osm2pgsql. For me Steve Coast lost his credibility when he joined the closed and proprietary what3words. [1] https://wi…
But you don't need to complicate the storage format to fix a problem like that. You can build validation tools that will check whether the stored data conforms to the correct specified geometry, and only emit valid polygons to later tools in the pipeline when they do.
"Be liberal in what you accept and strict in what you send" is still a good principle. The problem with rejecting invalid structures at the data storage format instead of a later validation step is that it hurts flexibility and extensibility. If later on you need a different type of polygon that would be rejected by the specification, you'll need to create a new version of the file format and update all tools reading it even if they won't handle the new type, instead of just having old tools silently ignoring the new format that they don't understand.