The lat/lon floating point delusion • Hacker News Board

Re: The lat/lon floating point delusion

#111

Post by rumanator » Fri, Aug 09, 2019, 5:19 PM UTC

falcolas wrote: post #54

jjoonathan wrote: post #20

Yes, why don't people spend their time doing the extra work to figure out the exact number of meaningful digits in their measurements when the defaults work just fine for their non-scientific, non-metrological purposes? A mystery for the ages.

Because the value, as presented, is wrong. "[but it's being used for] non-scientific, non-metrological purposes" which doesn't change that it's the wrong value. Imagine, for a moment, that someone uses these values to put up a fence. Is the fence on their property? Those crap values beyond the actual significant digits of the measurement may tell you that you are, when you're not. That will cost you time and money do…

> Because the value, as presented, is wrong.

That's not true nor is it the problem at all. The thesis was that the values, albeit correct, have too many significant digits, which in turn reflect differences which lie somewhere between having no practical use or being absurd.

> Imagine, for a moment, that someone uses these values to put up a fence.

That example is very poor. Any engineer can tell you that a measurement is meaningless without tolerances/margin of error, and the tolerance in effect when putting up a fence is not expressed in a microscopic scale.

Re: The lat/lon floating point delusion

#112

Post by thelittlenag » Fri, Aug 09, 2019, 5:29 PM UTC

grouseway wrote: post #105

thelittlenag wrote: post #104

These kinds of issues are what we've solved at QALocate. There are a number of different issues here, which I think are only partly explored through the article. Let's take as given that you need to direct a person some place. In the article, they are directing someone to a restaurant. But this gets complicate fast. Is this person a patron rather than, say, a deliver driver there to pick up an order, or a plumber the…

Is the Joe Pesci picture a joke or has your website been hacked? https://www.qalocate.com/company/

HAHAHAHA! Having not met Joe I don't actually know, but I presume that is a legit picture.

Re: The lat/lon floating point delusion

#113

Post by ramshorns » Fri, Aug 09, 2019, 5:30 PM UTC

rumanator wrote: post #111

falcolas wrote: post #54

Earlier quoted context omitted.

Because the value, as presented, is wrong. "[but it's being used for] non-scientific, non-metrological purposes" which doesn't change that it's the wrong value. Imagine, for a moment, that someone uses these values to put up a fence. Is the fence on their property? Those crap values beyond the actual significant digits of the measurement may tell you that you are, when you're not. That will cost you time and money do…

> Because the value, as presented, is wrong. That's not true nor is it the problem at all. The thesis was that the values, albeit correct, have too many significant digits, which in turn reflect differences which lie somewhere between having no practical use or being absurd. > Imagine, for a moment, that someone uses these values to put up a fence. That example is very poor. Any engineer can tell you that a measureme…

> a measurement is meaningless without tolerances/margin of error

Reporting a value with too many significant digits is the same as reporting an incorrect margin of error, which (arguably) makes the value wrong.

Re: The lat/lon floating point delusion

#114

Post by mark-r » Fri, Aug 09, 2019, 5:34 PM UTC

maxerickson wrote: post #13

remus wrote: post #8

I find it hard to get that worked up about this. The positions are accurate at least, and if you're really so hard up for space that saving a couple of digits is going to make all the difference then you can think about it for 5 mins and choose a precision/storage trade off that's appropriate for you.

The representation of the numbers doesn't convey any information about accuracy, and it's at least a good practice to store data in a way that conveys the precision it was gathered with.

Are you proposing doing away with IEEE 754? That's probably the origin of all these extra digits.

Re: The lat/lon floating point delusion

#115

Post by kgwgk » Fri, Aug 09, 2019, 5:36 PM UTC

dfranke wrote: post #58

People have this idea that when you take a measurement, you have so-and-so number of significant figures that are probably correct and the rest are just pure noise. But that's not how measurement error works. In the real world, physical measurement errors are more-or-less normally distributed (we don't have to argue about the "more-or-less" part because my argument here holds for any distribution other than a uniform…

> slightly more than half the time, 45.73490534578° is still going to be closer to the true value than 45.7349053457° is. By performing significant figure rounding,

That's not rounding, so I'll assume you meant "slightly more than half the time, 45.73490534578° is still going to be closer to the true value than 45.7349053458° is".

And that may be true, but if 50.0001% of the time one is closer to the true value than the other that's essentially meaningless.

Re: The lat/lon floating point delusion

#116

Post by lutorm » Fri, Aug 09, 2019, 5:39 PM UTC

mabbo wrote: post #101

dfranke wrote: post #58

People have this idea that when you take a measurement, you have so-and-so number of significant figures that are probably correct and the rest are just pure noise. But that's not how measurement error works. In the real world, physical measurement errors are more-or-less normally distributed (we don't have to argue about the "more-or-less" part because my argument here holds for any distribution other than a uniform…

The question isn't about mathematical rigor. It's about utility and distraction. In much of the world, phone numbers work even if you add extra numbers that aren't needed. So I could give you my phone number plus 10 extra digits that change nothing. The end result would be the same utility to you (you can contact me) but with an increased cost in recording, memorizing, chance of error. Using lat/lon to an unnecessary…

Exactly. Extraneous, useless, information is not no-value, it's negative-value, because of this.

Re: The lat/lon floating point delusion

#117

Post by donarb » Fri, Aug 09, 2019, 6:03 PM UTC

I've been working with the AWS Textract, it scans image files and returns the coordinates of the scanned text. The coordinates are percentages of the width/height of the image where one edge is 0.0 and the opposite edge is 1.0. The software returns numbers like "left = 0.3140530586242676", which means that the text's left coordinate is 31% from the edge. This is providing sub-pixel precision for 150 dpi documents.

Re: The lat/lon floating point delusion

#118

Post by frenchyatwork » Fri, Aug 09, 2019, 6:05 PM UTC

hamandcheese wrote: post #109

jgtrosh wrote: post #64

Earlier quoted context omitted.

I believe you're wrong. https://en.wikipedia.org/wiki/Significant_figures

I would never assume the displayed digits are significant in any non-scientific context.

People use significant digits colloquially, though with a lot less rigour. For example, If I say I'm 6 feet tall, people will probably guess that I'm between 5 ft 10 in and 6 ft 2 in. If I say I'm 5 ft 11 in, people will probably guess that I'm between 5 ft 10.5 in and 5 ft 11.5 in.

Re: The lat/lon floating point delusion

#119

Post by tony_cannistra » Fri, Aug 09, 2019, 6:10 PM UTC

wpietri wrote: post #79

In contrast to his 4-digits-are-fine notions, let me offer a counterexample: http://scissor.com/transient/destination_heatmap/ I am working on a Twitter bot called @sfships, which monitors the comings and goings of large ships in the San Francisco Bay: https://twitter.com/sfships As part of that, I generated the above map from AIS data. [1] It's basically where ships stop. If you zoom in on the San Francisco waterfro…

I used to have a job where I analyzed global ship tracking data (AIS and other less-open (i.e. more classified) data streams) for law enforcement and environmental conservation purposes, and learned one thing: man is AIS data a bitch.

This is especially true when you're dealing with potential bad actors who can spoof their transmissions, combined with incredibly poor vessel metadata databases and strange satellite AIS coverage issues.

If there are any data scientists out there looking for a real head-basher, start an AIS project.

Re: The lat/lon floating point delusion

#120

Post by maxerickson » Fri, Aug 09, 2019, 6:19 PM UTC

mark-r wrote: post #114

maxerickson wrote: post #13

Earlier quoted context omitted.

The representation of the numbers doesn't convey any information about accuracy, and it's at least a good practice to store data in a way that conveys the precision it was gathered with.

Are you proposing doing away with IEEE 754? That's probably the origin of all these extra digits.

Not at all. I'm suggesting people think about what they are doing. Blindly sticking numbers into a format that is designed to be computationally efficient is not a great way to store data. That doesn't mean that a computationally efficient representation is useless.