Live data from Hacker News

PostGIS – Spatial and Geographic Objects for PostgreSQL

postgis.net

21–30 of 89 posts

Re: PostGIS – Spatial and Geographic Objects for PostgreSQL

#21

FWIW, even though PostGIS is pretty great, if your use-case is primarily offline analysis and you don't need the data to be permanently accessible or writable, consider not using a database at all. You can do a lot, a lot faster with e.g. https://shapely.readthedocs.io/en/stable/manual.html and/or https://geopandas.org/ .

I'm not sure what 'a lot faster' really means in this context.

Honestly, I've found using Spatialite queries to be orders of magnitude faster for analysis than shapely or geopandas. The latter typically imply row-by-row selection and manipulation for starters.

If you can wrangle the data into a geopackage first it's super easy to run queries over the data and extract what you need.

Re: PostGIS – Spatial and Geographic Objects for PostgreSQL

#22
post #6

Here's an interesting performance comparison of PostgreSQL with PostGIS and MongoDB. PostgreSQL outperformed MongoDB in almost all their cases. https://link.springer.com/article/10.1007/s10707-020-00407-w I've decided to use PostgreSQL in my projects.

Thanks for sharing this! I've been thinking about alternatives to PostGIS to handle larger datasets (millions) and nothing seems to come close to the level of functionality, performance and community support of Postgres

Re: PostGIS – Spatial and Geographic Objects for PostgreSQL

#23

I've been using PostGIS a bit for a toy project with Elite: Dangerous star system data. It's been a hoot, but I do worry that I'm going to start having trouble optimizing my queries. Finding all the systems within say 20 Ly of our sun can take upwards of a few seconds, and I've already added a GIST index on the positions. I also find it a bit strange how 3D feels kinda tacked on, but it makes sense when you realize m…

My suggestion might be to a precalculated step to split "the universe" into grid areas divided by geometry complexity (i.e. population density gridding) then process these reference areas in parallel. How you do the latter is probably the interesting part!

Re: PostGIS – Spatial and Geographic Objects for PostgreSQL

#24

FWIW, even though PostGIS is pretty great, if your use-case is primarily offline analysis and you don't need the data to be permanently accessible or writable, consider not using a database at all. You can do a lot, a lot faster with e.g. https://shapely.readthedocs.io/en/stable/manual.html and/or https://geopandas.org/ .

I'm not sure what 'a lot faster' really means in this context. Honestly, I've found using Spatialite queries to be orders of magnitude faster for analysis than shapely or geopandas. The latter typically imply row-by-row selection and manipulation for starters. If you can wrangle the data into a geopackage first it's super easy to run queries over the data and extract what you need.

Faster both in terms of querying and in terms of doing the kind of analysis you want and getting the answers you need.

Of course, happy to acknowledge that different tools might work better in different scenarios. For example, I suspect that speed of querying is really just due to the data being in memory so if you can configure Spatialite or PostGIS to do the same, I certainly wouldn't be surprised if you say you can do even better.

But for one-off analyses, it's common to spend a lot of time just getting your data into the right shape, doing various manipulations, perhaps even wrangling the geometries. For that, working entirely within SQL is frustrating as heck. For example, I did an analysis on flight paths over heavily populated areas once, which involved turning infrequent point locations with gaps in the data into a smooth interpolated flight path. That's easy if you have numpy and scipy at your disposal, otherwise it's not. Another analysis involved estimating housing prices in neighborhoods without any recent sales, from prices in adjacent neighborhoods with sales, and again it's easy to code up an algorithm to fill the gaps or to run a geostatistical analysis that can impute the missing values, but not if all you have is SQL, or if you have to constantly do roundtrips between database and code.

I mention all this not to start an argument, but simply because when I first started doing GIS work, I was very confused about what the right tools and workflow were, and once I embraced projections (vs. working directly with spheroids) and in-memory analysis in Python, my productivity went way up. If other people find themselves in the same scenario, they owe it to themselves to try out both approaches to see what works best for them.

Re: PostGIS – Spatial and Geographic Objects for PostgreSQL

#25
post #12

4 years ago one of my clients wanted to "donate" a system for the local fire department to help them do a quick proximity search to find the fire hydrants and quickly choose the healthy one near the fire. And since it was charity and had a bunch of private data Google was not an option ($$$$), so I (just a full-stack developer back then) was like "listen I have no idea what is this GIS stuff, but I'll give it a try",…

Comparatively, I've spent 4 years of my life on a CompSci degree + 2 years on a GIS Advanced Diploma achieved over 10 years ago mind you). There was a point in my life a few years ago where my study was so unnoticed and under appreciated that I gave it up and now I just do full project management - and I barely even do that.

I blame the internet and the accessibility of knowledge to the point where people can go "IDK WTF this is, but lemme google it".

Seriously, 6 years of my fucking life dedicated to something that is specialist knowledge and it's just now garbage.

so, good on you for winging it and ruining other's careers - lol

Re: PostGIS – Spatial and Geographic Objects for PostgreSQL

#26
post #12

4 years ago one of my clients wanted to "donate" a system for the local fire department to help them do a quick proximity search to find the fire hydrants and quickly choose the healthy one near the fire. And since it was charity and had a bunch of private data Google was not an option ($$$$), so I (just a full-stack developer back then) was like "listen I have no idea what is this GIS stuff, but I'll give it a try",…

Had a similar career shift after helping local public health authorities map COVID-19!

Re: PostGIS – Spatial and Geographic Objects for PostgreSQL

#27
post #25
post #12

4 years ago one of my clients wanted to "donate" a system for the local fire department to help them do a quick proximity search to find the fire hydrants and quickly choose the healthy one near the fire. And since it was charity and had a bunch of private data Google was not an option ($$$$), so I (just a full-stack developer back then) was like "listen I have no idea what is this GIS stuff, but I'll give it a try",…

Comparatively, I've spent 4 years of my life on a CompSci degree + 2 years on a GIS Advanced Diploma achieved over 10 years ago mind you). There was a point in my life a few years ago where my study was so unnoticed and under appreciated that I gave it up and now I just do full project management - and I barely even do that. I blame the internet and the accessibility of knowledge to the point where people can go "IDK…

Not his or her fault the tooling obseleted the specialty knowledge.

Re: PostGIS – Spatial and Geographic Objects for PostgreSQL

#28

Earlier quoted context omitted.

I'd argue that being able to use sql to more easily combine ans filter datasets, including non-geospatial ones is still very useful in the circumstances you described.

Not really, combining datasets in (Geo)Pandas is very straightforward, including spatial joins: https://geopandas.org/docs/user_guide/mergingdata.html#spati... Of course, it's all a matter of personal preference, but I have used both PostGIS and GeoPandas extensively.

You will eventually run out of memory or some other restraint given large enough datasets. PostGIS will churn through it and not crash...

Re: PostGIS – Spatial and Geographic Objects for PostgreSQL

#29
post #8

PostGIS is one of those rare examples of highly specialised software that is both OSS and best-in-class. Usually OSS is relegated for highly commoditized software but GIS is anything but even after all these years. I can't really think of many others? Maybe OptaPlanner would be another candidate.

Is OptaPlanner best in class? (No snark) — It might be the best in Java but not sure more broadly.
Post reply on HN