Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tile38 – Realtime geofencing and geospatial index, v1.7.0

github.com

Post reply on HN
Print view View original

21–29 of 29 posts

Re: Tile38 – Realtime geofencing and geospatial index, v1.7.0

#21

Post by mynewtb »

tidwall wrote: post #8
Bedon292 wrote: post #6

Earlier quoted context omitted.

Thanks for the response. I see what I was actually looking for was on http://tile38.com/topics/object-types/ Right there on the front page, and not in the docs tab where I was looking. For the GeoJSON, does it support properties? Meaning can I store a a whole document in there? Or would it be best to just store an ID and lookup the ID in a DB or something?

> For the GeoJSON, does it support properties? Yes, Tile38 has full GeoJSON support including properties for Feature objects. Though there is one exception, the CRS key is ignored because WGS84 is assumed. > Meaning can I store a a whole document in there? Yes, as long as the document conforms to http://geojson.org/geojson-spec.html . Though JSON keys that are not defined in the spec will be stripped out of the docum…

The RFC also limits GeoJSON to WGS84 by the way. https://tools.ietf.org/html/rfc7946

Re: Tile38 – Realtime geofencing and geospatial index, v1.7.0

#22

Post by tidwall »

mynewtb wrote: post #21
tidwall wrote: post #8

Earlier quoted context omitted.

> For the GeoJSON, does it support properties? Yes, Tile38 has full GeoJSON support including properties for Feature objects. Though there is one exception, the CRS key is ignored because WGS84 is assumed. > Meaning can I store a a whole document in there? Yes, as long as the document conforms to http://geojson.org/geojson-spec.html . Though JSON keys that are not defined in the spec will be stripped out of the docum…

The RFC also limits GeoJSON to WGS84 by the way. https://tools.ietf.org/html/rfc7946

Thanks for sharing. The draft RFC at the beginning of the year included the CRS key so it must've be a recent decision.

Re: Tile38 – Realtime geofencing and geospatial index, v1.7.0

#23

Post by Fiahil »

tidwall wrote: post #16
Fiahil wrote: post #15

Neat! Does someone already used it in production? I'm wondering if it could be useful to replace a medium sized postgres database. I'm trying to replace some clustering currently done with Elasticsearch with postgis, but it's really slow (6m points * 700 boxes, it's taking 20 minutes to scan the table).

Yes, it is being used in production. General purpose spatial queries with Tile38 are very fast. Likely much quicker than PostGIS, but PostGIS will be able to store much more data. It's a tradeoff for sure. If your dataset isn't terribly big and performance is what you're after then I recommend giving Tile38 a try.

Thank you for the details! After all, if I can manage to fit around 6 million points into a single r4.large and get a decent response time when clustering them, I can't see why I shouldn't try something with it as a secondary data store.

Re: Tile38 – Realtime geofencing and geospatial index, v1.7.0

#24

Post by tidwall »

Fiahil wrote: post #23
tidwall wrote: post #16

Earlier quoted context omitted.

Yes, it is being used in production. General purpose spatial queries with Tile38 are very fast. Likely much quicker than PostGIS, but PostGIS will be able to store much more data. It's a tradeoff for sure. If your dataset isn't terribly big and performance is what you're after then I recommend giving Tile38 a try.

Thank you for the details! After all, if I can manage to fit around 6 million points into a single r4.large and get a decent response time when clustering them, I can't see why I shouldn't try something with it as a secondary data store.

I can't argue with that. :)

Re: Tile38 – Realtime geofencing and geospatial index, v1.7.0

#25

Post by nrjdhsbsid »

tidwall wrote: post #20
nrjdhsbsid wrote: post #18

Earlier quoted context omitted.

Lol looks like this database needs its own database. Yeah seems like it should be a plugin or library not sure why it's standalone if it's just backed by redis.

It's not backed by Redis. It's fully standalone. I wish I could have made a redis module back when I started the project, but I'm plenty happy how Tile38 has matured.

Eh, fair enough then. If you don't have to setup redis yourself it doesn't really matter

Re: Tile38 – Realtime geofencing and geospatial index, v1.7.0

#26

Post by bpicolo »

3adawi wrote: post #10

how fast is within? does it use ray tracing?

Depends on which method you use I assume. For geohashing, WITHIN should be best case O(1) and worst case O(n) depending on specificity you care about, where n is the number of geohash cells you need to generate to match the bounding box you care about (which is typically pretty small unless you want, like, a 1km box with 1cm granularity)

Frontloading computation into generating smart data structures give you better choices here.

Re: Tile38 – Realtime geofencing and geospatial index, v1.7.0

#27

Post by tidwall »

Opteron67 wrote: post #14
tidwall wrote: post #5

Earlier quoted context omitted.

> I see that it is in Memory, but are there any practical limits? Yes. The in-memory database is basically a big btree + a big rtree. These trees contain many pointers to geo objects and depending on the complexity of the objects, it may take up a lot of memory. A 16GB machine can typically store 100 million+ points. > How much memory is recommended for what size datasets? Depends on the type of object and the length…

> but I would conservatively use 128-bytes per point as a general rule. why not store close items together in a small block and use offsets + delta encoding ?

Thanks for the suggestion, I'll need to investigate further. Right now, under the hood Tile38 uses an 3d rtree. Each node clusters 8-16 objects together. I'm always looking for a better way.

Re: Tile38 – Realtime geofencing and geospatial index, v1.7.0

#28

Post by tidwall »

3adawi wrote: post #10

how fast is within? does it use ray tracing?

Sorry, my previous answer was not very descriptive.

The WITHIN call will be on average O(log(N)) because all objects are stored in an rtree. But it's usually very very fast to locate objects based on their outermost bounding area. The complexity of the object will add to the duration. Simple points are super fast, like sub microsecond fast in many cases. A MultiPolygon can be much slower.

Tile38 uses a modified raycasting algorithm, not raytracing for it's polygon detections.

Re: Tile38 – Realtime geofencing and geospatial index, v1.7.0

#29

Post by [deleted] »

tidwall wrote: post #16
Fiahil wrote: post #15

Neat! Does someone already used it in production? I'm wondering if it could be useful to replace a medium sized postgres database. I'm trying to replace some clustering currently done with Elasticsearch with postgis, but it's really slow (6m points * 700 boxes, it's taking 20 minutes to scan the table).

Yes, it is being used in production. General purpose spatial queries with Tile38 are very fast. Likely much quicker than PostGIS, but PostGIS will be able to store much more data. It's a tradeoff for sure. If your dataset isn't terribly big and performance is what you're after then I recommend giving Tile38 a try.

[deleted]
Post reply on HN