61–70 of 78 posts
You can save a lot of RAM by using an array of interleaved coordinates. For an additional bonus, you can also compress rings by storing the ring offsets inside a larger array.
One task where GeoJSON falls down is simplification of a group of polygons with common boundaries, e.g. the 48 conterminous US states. If you start with a highly detailed set of polygons, you need to simplify them for practical display in an online map. GeoJSON doesn't encode the fact that the boundary points are common between adjacent polygons. When you simplify those polygons, each one is handled separately, and y…
I like TopoJSON and have used it in projects. But it's weird to set it up as opposition to GeoJSON. It's a complement. GeoJSON is a general data format meant to replace uses of ESRI Shapefiles and other complex formats. TopoJSON is more of a solution for a particular application need. Is there much work developing or using TopoJSON these days? I haven't seen much about it in a few years.
I'm just saying that for the specific task I mentioned GeoJSON or any format such as shapefiles that store polygons individually naturally leads to the "sliver" problem.
A nice processing pipeline is:
1. Convert GeoJSON to TopoJSON.
2. Run the simplification on the TopoJSON.
3. Convert the resulting TopoJSON back to GeoJSON.
The TopoJSON GitHub has tools for each of these steps.
One task where GeoJSON falls down is simplification of a group of polygons with common boundaries, e.g. the 48 conterminous US states. If you start with a highly detailed set of polygons, you need to simplify them for practical display in an online map. GeoJSON doesn't encode the fact that the boundary points are common between adjacent polygons. When you simplify those polygons, each one is handled separately, and y…
GeoJSON is not TopoJSON. Saying that is "falling down" is like criticizing a zebra for not being a giraffe. GeoJSON is a mapping of the (non-topological) "simple features" model into JSON, full stop. It does that fine.
I don't think I would trust a zebra or a giraffe for this task either.
Earlier quoted context omitted.
Is this actually GeoJSON falling down, or decades of convention extended to JSON? Topology is great, but it is sidestepped by Shapefile/WKT/WKB/etc, in favor of independent primitives like POINT, LINE, POLYGON. If GeoJSON did not exist as a new JSON GIS data format encoding these primitives, TopoJSON would not have "replaced" it, due to the added mis-match with other non-topological formats. From what I can tell, the…
Right. Encoding a union algorithm into the data structure just introduces the reverse problem: Selecting a subset now requires extra logic beyond jq.
Earlier quoted context omitted.
The problem is simplification. Suppose two regions share a border with some nonlinear points a, b, c, d. Simplifying the polygon for the first region might yield a, b, d while the second yield a, c, d. This creates gaps or overlaps between the two regions.
But what is the border? Set the border to what it actually is, not a simplification of it. The state of Colorado is formally a 697 sided polygon, don't simplify it to a rectangle.
https://maps.app.goo.gl/JH93ko96QcoLXuBJ9
https://maps.app.goo.gl/au53iTnsmNdFuEZV8
Even the one zoomed in on the state appears to use maybe 15-20 vertices max.
In the second one, if I squint real hard I can just barely make out one slight dogleg on the western border and one on the south. And that is partly because I knew to look for them in the zoomed-in map.
If we use, say, the Census TIGER/Line boundary definitions for the states, we are probably talking about hundreds of thousands of vertices, perhaps millions. You won't be using those in an online map without simplifying.
Earlier quoted context omitted.
This is not what OP is describing. It is very common to simplify objects for decreasing boundary objects by orders of magnitude. GeoJSON is missing correlation when you do that. Simplifying country objects from a GeoJSON source could lead to a gap between the country borders. So you either have poor representation or a longer pipeline to convert objects to an amenable object set. It also breaks idempotency in some re…
To do the simplification, you detect shared borders, simplify and generate polygons again. That doesn’t make topojson inherently superior. You can convert back and forth and for many applications geojson is easier to process.
Interesting but, IMO, probably one of the worst uses of JSON. The data you would want to consume is already not "human readable" so it instead introduces a lot of bloat for really no benefit. If you have a non-insignificant amount of data points to track this is going to eat just a ton of memory while also being pretty slow to encode/decode. Imagine, for example, if we encoded this as a binary. First 2 bytes for the…
This is a fair critique, however, for any large GeoJSON, the coordinate arrays will dominate the size. I think it's also safe to assume this data will be gzipped at rest and over the wire, which will eliminate most of the "header" metadata size you mention. As you point out, it would be much more efficient to have a binary format, and there are good examples like these, that are ~2-3x smaller in benchmarks:
https://flatgeobuf.org/ https://github.com/mapbox/geobuf
That said, I think GeoJSON should be compared against other human readable formats like KML, which has a lot of wasted space as well, while being more difficult to read/write.
Earlier quoted context omitted.
JSON parsing is probably one of the most thoroughly optimized subsystems in the whole industry at this point. Obviously there are ways to encode the same data that are easier to parse (e.g. instead of absolute floating point coordinates, use integer deltas along a path in some reasonable CRS) but because this inefficient representation is so common and so long-standing the parsing is faster than people think.
the default parsers all load the entire thing in mem, which is not good. so you need a stream-based parser, which nobody does an effort to write/use for json. especially since geojson is a web format, and people just default to json.parse, which is blocking. and even then, even if you did use the custom one, it likely won't be a geojson-tailored one, so because key-order isn't guaranteed, any parser for geo-json will…
Earlier quoted context omitted.
To do the simplification, you detect shared borders, simplify and generate polygons again. That doesn’t make topojson inherently superior. You can convert back and forth and for many applications geojson is easier to process.
Yes, you could write code to do that. Or use the utilities provided in the TopoJSON GitHub and let them do it for you: convert to TopoJSON, simplify, convert back to GeoJSON. They have already written all the code for you.
Earlier quoted context omitted.
The problem is simplification. Suppose two regions share a border with some nonlinear points a, b, c, d. Simplifying the polygon for the first region might yield a, b, d while the second yield a, c, d. This creates gaps or overlaps between the two regions.
So don’t simplify the shapes on their own. Geojson is a storage and exchange format, you can still convert it to other formats if you want to modify it.
20-25 years ago I worked a lot with map data from otherwise high quality, and sometimes authoritative, sources like the USGS and NOAA that had this non-identical shared boundaries problem (in formats other than GeoJSON). If the format doesn't allow such mistakes to be expressed, then they have to fix their data to publish it in said format.