Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The lat/lon floating point delusion

datafix.com.au

Post reply on HN
Print view View original

71–80 of 154 posts

Re: The lat/lon floating point delusion

#71

Post by moron4hire »

I recently ran into a weird error concerning Lat/Lngs and floating point numbers. Google has a service for retrieving imagery from Maps, and you can even request that imagery have certain paths drawn on it. You encode those paths as part of your request URI, but if you have a lot of points, you could end up exceeding their maximum URI length restriction. So they also define a hashing system for compressing those points into a single value, which is defined here: https://developers.google.com/maps/documentation/utilities/p...

One of the things they don't mention is that the rounding step does not work as they expect for single-precision floats. You have to use double-precision floats to get the same results that they are demonstrating in the example.

They are asking for 5 values to the right of decimal points, which, with the maximum of 3 digits to the left of the decimal point, means a total of 8 significant digits. Single-precision floats should be good to 9 digits, but the rounding step is off by one when using singles.

The solution was to cast singles to doubles before performing the rounding. Which seems absurd.

Single-precision floats should be enough precision for lat/lngs anywhere on the Earth, for everything other than some applications in commercial-grade surveying. And if that is the job you're doing, you shouldn't be using Google Maps.

Re: The lat/lon floating point delusion

#72

Post by whatshisface »

Majromax wrote: post #19
whatshisface wrote: post #12

Earlier quoted context omitted.

Error tends to grow as the calculation proceeds. 6th-decimal "insignificant rounding" done over and over as the calculation proceeds can grow to become arbitrarily large.

What numerically unstable algorithms are ever performed with geolocations? Moreover, given the author's point that real measurement errors exceed the false precision of published data, if such a calculation were performed and did provide "arbitrarily large" error, it would indicate that the result should in fact be nonsense.

I'm thinking about the case where a calculation extends across many roundings.

Re: The lat/lon floating point delusion

#73

Post by moron4hire »

bytematic wrote: post #31

What about future GIs programs that can handle these digits, should we not futureproof anything that doesn't exactly utilize it today?

It's not about software that can't handle the precision (the software can). It's about the message the false precision conveys. By stating a value out to that many decimal places, they are stating that the location is good to a millimeter. But the location was almost certainly captured by a GPS device that can only locate you down to a 5 meter radius. They've provided a value that implies a point, when they should be providing a value that defines a circle.

Re: The lat/lon floating point delusion

#74

Post by jjoonathan »

falcolas wrote: post #54
jjoonathan wrote: post #20

Yes, why don't people spend their time doing the extra work to figure out the exact number of meaningful digits in their measurements when the defaults work just fine for their non-scientific, non-metrological purposes? A mystery for the ages.

Because the value, as presented, is wrong. "[but it's being used for] non-scientific, non-metrological purposes" which doesn't change that it's the wrong value. Imagine, for a moment, that someone uses these values to put up a fence. Is the fence on their property? Those crap values beyond the actual significant digits of the measurement may tell you that you are, when you're not. That will cost you time and money do…

It's wrong if you ignore context -- but then, everything is.

The context is that it's in a database serving up business locations. How is the geographic area owned (or occupied?) by a business reduced to a point? They don't say, so it's clearly unreasonable to expect accuracy and precision beyond, say, property boundaries, and perhaps not even that.

It would be madness to infer that the extra digits denote precision beyond the crude bounds set by context.

It would be further madness to reverse engineer a property boundary -- an entire path of points -- from a single point in a database.

Re: The lat/lon floating point delusion

#75

Post by dfranke »

protonfish wrote: post #65
dfranke wrote: post #58

People have this idea that when you take a measurement, you have so-and-so number of significant figures that are probably correct and the rest are just pure noise. But that's not how measurement error works. In the real world, physical measurement errors are more-or-less normally distributed (we don't have to argue about the "more-or-less" part because my argument here holds for any distribution other than a uniform…

The problem is that these lat/lon figures are not displayed with precision information at all, so we can't even know the std dev. Sig figs are a simple and clear way to communicate the precision of a measurement, but if you want to be more statistically accurate, you can use parentheses to indicate the standard deviation. In your example (if I mess this up please correct me, but I think) it would be 45.73490534578(10…

I've never seen the convention you're using and I don't think I understand it. Conventions I've seen include:

* Give an error bound like 45.73490534578° (±0.00002°) and indicate in prose that this is a 2σ bound.

* Put non-significant figures in parenthesis, like 45.73490(534578)° (EDIT: possibly I've misinterpreted this one when I've seen it, see logfromblammo's reply)

* Put a bar over the last significant figure, like 45.73490̄534578 (hopefully this one renders properly when I post this... (EDIT: nope))

Re: The lat/lon floating point delusion

#76

Post by falcolas »

izacus wrote: post #60
falcolas wrote: post #54

Earlier quoted context omitted.

Because the value, as presented, is wrong. "[but it's being used for] non-scientific, non-metrological purposes" which doesn't change that it's the wrong value. Imagine, for a moment, that someone uses these values to put up a fence. Is the fence on their property? Those crap values beyond the actual significant digits of the measurement may tell you that you are, when you're not. That will cost you time and money do…

These values are as right as setting least significant digits to 0 would be. The sensor has limited precision and the other digits might as well be 0 or 584985948 and the value would be no more precise.

If you're given 8 significant digits, how many of those are correct? 3? 4? 5? The difference between those three is the difference between a position on a street block and the position of a person in a room (to quote XKCD). At least with 0's, you can approximate how accurate the value is. With garbage, all you can do is hope you've guessed correctly.

Re: The lat/lon floating point delusion

#77

Post by 7952 »

falcolas wrote: post #54
jjoonathan wrote: post #20

Yes, why don't people spend their time doing the extra work to figure out the exact number of meaningful digits in their measurements when the defaults work just fine for their non-scientific, non-metrological purposes? A mystery for the ages.

Because the value, as presented, is wrong. "[but it's being used for] non-scientific, non-metrological purposes" which doesn't change that it's the wrong value. Imagine, for a moment, that someone uses these values to put up a fence. Is the fence on their property? Those crap values beyond the actual significant digits of the measurement may tell you that you are, when you're not. That will cost you time and money do…

A lot of these kind of coordinates have no right answer anyway. They are just an abstract approximation of something else, like a parcel, address, or place to eat. And the lat/long will be completely useless for the vast majority of people.

In any serious application you should have known error values and probably be using a more appropriate projection. If you need to accurately locate something you need someone trained to do that with the right equipment.

Re: The lat/lon floating point delusion

#78

Post by falcolas »

jjoonathan wrote: post #74
falcolas wrote: post #54

Earlier quoted context omitted.

Because the value, as presented, is wrong. "[but it's being used for] non-scientific, non-metrological purposes" which doesn't change that it's the wrong value. Imagine, for a moment, that someone uses these values to put up a fence. Is the fence on their property? Those crap values beyond the actual significant digits of the measurement may tell you that you are, when you're not. That will cost you time and money do…

It's wrong if you ignore context -- but then, everything is. The context is that it's in a database serving up business locations. How is the geographic area owned (or occupied?) by a business reduced to a point? They don't say, so it's clearly unreasonable to expect accuracy and precision beyond, say, property boundaries, and perhaps not even that. It would be madness to infer that the extra digits denote precision…

You can click at arbitrary positions on those maps and get 6+ sig-fig values for those positions. And since the map displays a satellite representation of the property, is it really that extraordinary that people might consider it to be sufficiently precise enough to put in a fence?

FWIW, the error margin for 6 significant digits is around 11 cm. Accurate enough to point at a computer mouse.

Re: The lat/lon floating point delusion

#79

Post by wpietri »

In contrast to his 4-digits-are-fine notions, let me offer a counterexample: http://scissor.com/transient/destination_heatmap/

I am working on a Twitter bot called @sfships, which monitors the comings and goings of large ships in the San Francisco Bay: https://twitter.com/sfships

As part of that, I generated the above map from AIS data. [1] It's basically where ships stop. If you zoom in on the San Francisco waterfront, you will see a grid of dots. That's because the AIS protocol stores lat/lon as minutes/10000 [2], throwing away more detailed information.

This is adequate for its initial intended purpose, which is putting ships on radaresque displays so that ships don't hit one another, etc. But it produces all sorts of artifacts and issues when one tries to use the data more broadly.

And in case anybody is interested in playing with this, I have written a python parser for AIS data with a bunch of Unixy command line tools, including one that just turns the weird 90s protocol into more modern JSON: https://github.com/wpietri/simpleais

[1] https://en.wikipedia.org/wiki/Automatic_identification_syste...

[2] https://gpsd.gitlab.io/gpsd/AIVDM.html#_types_1_2_and_3_posi...

Re: The lat/lon floating point delusion

#80

Post by falcolas »

7952 wrote: post #77
falcolas wrote: post #54

Earlier quoted context omitted.

Because the value, as presented, is wrong. "[but it's being used for] non-scientific, non-metrological purposes" which doesn't change that it's the wrong value. Imagine, for a moment, that someone uses these values to put up a fence. Is the fence on their property? Those crap values beyond the actual significant digits of the measurement may tell you that you are, when you're not. That will cost you time and money do…

A lot of these kind of coordinates have no right answer anyway. They are just an abstract approximation of something else, like a parcel, address, or place to eat. And the lat/long will be completely useless for the vast majority of people. In any serious application you should have known error values and probably be using a more appropriate projection. If you need to accurately locate something you need someone trai…

They aren't approximations; you can verify this yourself. Pull up Google Maps, pick an arbitrary spot with no businesses, and click. You will get at least 6 significant digit coordinates.
Post reply on HN