Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In Defense of OpenStreetMap's Data Model

stevecoast.substack.com

Post reply on HN
Print view View original

51–60 of 130 posts

Re: In Defense of OpenStreetMap's Data Model

#51

Post by delusional »

stevage wrote: post #13

In the first part of the article I was thinking, oh, maybe Steve Coast isn't such a jerk after all. Then I got to the meat of it. Oh dear. As one of the many many people who has had to deal with OSM data, I curse people with this attitude that the mess is somehow desirable or necessary. It's not. There is a long spectrum between totally free form and completely constrained, and OSM's data model is painfully down the…

I tend to agree with you, having done a fair bit of cursing at the OSM format as well.

Yet they've made an open source map, and I haven't. The data tells me that I'm wrong.

Re: In Defense of OpenStreetMap's Data Model

#52

Post by maxerickson »

delusional wrote: post #51
stevage wrote: post #13

In the first part of the article I was thinking, oh, maybe Steve Coast isn't such a jerk after all. Then I got to the meat of it. Oh dear. As one of the many many people who has had to deal with OSM data, I curse people with this attitude that the mess is somehow desirable or necessary. It's not. There is a long spectrum between totally free form and completely constrained, and OSM's data model is painfully down the…

I tend to agree with you, having done a fair bit of cursing at the OSM format as well. Yet they've made an open source map, and I haven't. The data tells me that I'm wrong.

For a crowd sourced dataset, a strict ontology anyway wouldn't work. Instead of messy tag definitions you'd have tag use that didn't align with the definitions.

I don't mean that as an argument against improving the tagging!

The biggest friction point is probably that people resist rationalization of tagging schemes that have demonstrated themselves to be problematic.

The tagging system in the iD editor tries to address the issue, supporting search terms and suggesting related tags and so on.

The article is more about the underlying storage of the geometries (I don't think there is the same level of interest in changing the basic approach to tagging/categorization).

Re: In Defense of OpenStreetMap's Data Model

#53

Post by arccy »

TuringTest wrote: post #10
RicoElectrico wrote: post #8

The proposed improvements would obsolete a bunch of problems such as broken polygons [1] which happen regularly. They would also make processing OSM more accessible without needing to randomly seek over GBs of node locations just to assemble geometries which takes a significant runtime percentage of osm2pgsql. For me Steve Coast lost his credibility when he joined the closed and proprietary what3words. [1] https://wi…

But you don't need to complicate the storage format to fix a problem like that. You can build validation tools that will check whether the stored data conforms to the correct specified geometry, and only emit valid polygons to later tools in the pipeline when they do. "Be liberal in what you accept and strict in what you send" is still a good principle. The problem with rejecting invalid structures at the data storag…

postel was wrong https://tools.ietf.org/id/draft-thomson-postel-was-wrong-03....

Re: In Defense of OpenStreetMap's Data Model

#54

Post by seoaeu »

TuringTest wrote: post #10
RicoElectrico wrote: post #8

The proposed improvements would obsolete a bunch of problems such as broken polygons [1] which happen regularly. They would also make processing OSM more accessible without needing to randomly seek over GBs of node locations just to assemble geometries which takes a significant runtime percentage of osm2pgsql. For me Steve Coast lost his credibility when he joined the closed and proprietary what3words. [1] https://wi…

But you don't need to complicate the storage format to fix a problem like that. You can build validation tools that will check whether the stored data conforms to the correct specified geometry, and only emit valid polygons to later tools in the pipeline when they do. "Be liberal in what you accept and strict in what you send" is still a good principle. The problem with rejecting invalid structures at the data storag…

> "Be liberal in what you accept and strict in what you send" is still a good principle.

No, it is a terrible principle which produces brittle software and impossible to implement standards. The problem is that no one actually follows the “be strict in what you send” part, and just goes with whatever cobbled together mess the other existing software seems to accept. Before long, a spec compliant implementation can’t actually understand any of the messages that are being sent

> just having old tools silently ignoring the new format that they don't understand.

This sounds like another headache. I don’t want my tools silently breaking.

Re: In Defense of OpenStreetMap's Data Model

#55

Post by lovemenot »

Grimburger wrote: post #46
joshxyz wrote: post #35

Earlier quoted context omitted.

why not use the universal date format [1] that works for everyone? yyyy mm dd mm yyyy [1] https://twitter.com/dan_abramov/status/1447710863960551433

Unix time is the real universal date format surely? It underpins basically everything in datetime database entries. The problem is that most people can't read that along with the French somehow never being able to convince the world to adopt decimal time.

Why make a good point and follow it with a bad joke which: both has a class of victim and does not work as a joke?

Nobody has ever heard of "the French" undertaking such a project. And if it had ever happened, it would most surely have been a particular Academy or the like. Not the 65M people under your careless swipe.

Re: In Defense of OpenStreetMap's Data Model

#56

Post by seoaeu »

TuringTest wrote: post #7
MontyCarloHall wrote: post #5

Earlier quoted context omitted.

The entire article can be summed up as: “OSM stores maps as graphs, in flat files where each line is either a node, an ordered list of nodes, or metadata. The graph nodes can be arbitrarily ordered in OSM files, which leads to computational complexity when parsing them. This is not a bad thing, since it means that the spec for OSM files can be extremely simple, which makes it easy for people to contribute to OSM. Oth…

> OSM stores maps as graphs, in flat files where each line is either a node, an ordered list of nodes, or metadata. The graph nodes can be arbitrarily ordered in OSM files, which leads to computational complexity when parsing them. This is not a bad thing, since it means that the spec for OSM files can be extremely simple, which makes it easy for people to contribute to OSM. That's actually a sensible design. Treat u…

Wait, the proposed solution to a data format being slow to parse is to work around the bad performance by caching the already parsed representation? That seems like it has a clear flaw if you’re only accessing the data once…

Re: In Defense of OpenStreetMap's Data Model

#57

Post by deathanatos »

Grimburger wrote: post #46
joshxyz wrote: post #35

Earlier quoted context omitted.

why not use the universal date format [1] that works for everyone? yyyy mm dd mm yyyy [1] https://twitter.com/dan_abramov/status/1447710863960551433

Unix time is the real universal date format surely? It underpins basically everything in datetime database entries. The problem is that most people can't read that along with the French somehow never being able to convince the world to adopt decimal time.

> Unix time is the real universal date format surely?

Heh… It can't represent UTC (leap seconds). Implementation weirdness like not being able to represent dates outside the 32-bit second counter range due to using a 32-bit second counter. The occasional need to store the TZ the timestamp is relevant in/to.

> It underpins basically everything in datetime database entries.

It does do that.

> The problem is that most people can't read that

This is why I prefer RFC 3339 or ISO, in that order.

> along with the French somehow never being able to convince the world to adopt decimal time.

Somehow. :)

Re: In Defense of OpenStreetMap's Data Model

#58

Post by deathanatos »

lovemenot wrote: post #55
Grimburger wrote: post #46

Earlier quoted context omitted.

Unix time is the real universal date format surely? It underpins basically everything in datetime database entries. The problem is that most people can't read that along with the French somehow never being able to convince the world to adopt decimal time.

Why make a good point and follow it with a bad joke which: both has a class of victim and does not work as a joke? Nobody has ever heard of "the French" undertaking such a project. And if it had ever happened, it would most surely have been a particular Academy or the like. Not the 65M people under your careless swipe.

https://en.wikipedia.org/wiki/Decimal_time#France

Re: In Defense of OpenStreetMap's Data Model

#59

Post by Archelaos »

atoav wrote: post #31
zigzag312 wrote: post #24

Author states: >And that’s the point, rules and complexity have completely unknowable downsides. Downsides like the destruction of the whole project. With each rule and added complexity you make the system less human and less fun. You make it a Computer Scientists rube goldberg machine while sterilizing it of all the joy of life. While too much rules and complexity can certainly be bad, some basic amount of standardi…

The date example is a good one. No one has fun by choosing their own date format. This is putting the burden of choice onto the user. They might like to think about some map stuff and now they have to think about data format stuff. Of course projects like these have to strike a balance between the strictest bureaucratic nightmare and such a structure so loose that people are overburdened by the available options at e…

> The date format in the backend should be fixed and then you should offer flexibility in the frontend for user input.

Agreed. However, it might be not so easy for historical dates, because doing it correctly requires great diligence on the part of the tool developer as well as from the user to choose the correct calendar system. For example:

  Q: What is the correct representation of the date of Caesar's death, 15 March 44 BC in ISO-8601?

  A: -0043-03-13

  Why? -- Ancient dates are typically given according to the Julian calendar excluding a year 0, but ISO-8601 uses a proleptic Gregorian calendar including a year 0.
Post reply on HN