Live data from Hacker News

The lat/lon floating point delusion

datafix.com.au

31–40 of 154 posts

Re: The lat/lon floating point delusion

#32
post #18

Earlier quoted context omitted.

Error tends to grow as the calculation proceeds. 6th-decimal "insignificant rounding" done over and over as the calculation proceeds can grow to become arbitrarily large.

Which calculation on lat/lon coordinates would make an atom-scale error grow, say, a million times to millimeter-size?

"Build the gravitational wave observatory between the Thai restaurant and the railway station"

Re: The lat/lon floating point delusion

#34

Yes, why don't people spend their time doing the extra work to figure out the exact number of meaningful digits in their measurements when the defaults work just fine for their non-scientific, non-metrological purposes? A mystery for the ages.

I like your argument, but it would be nice to keep things civil.

Re: The lat/lon floating point delusion

#35
post #21

Does anyone here know what's the actual precision that we can get by the available equipment?

I know what I can get on a Nexus 6P and it is not a pleasant number. To make things worse the logic calculating the accuracy is also broken (e.g. a GPS point stating 5m accuracy may jump around by tens of meters). tl;dr cheap phone gps are not to be trusted without a lot of vetting

Re: The lat/lon floating point delusion

#36
post #2

Did the author think about the implementation? If we assign semantic meaning to the number of decimal points, now we also need to store the number of significant digits side by side with the actual number. We shouldn't round off internally because we might use it in as an intermediate result in a calculation.

I would consider storing (as part of the database) or defining (as part of the display code) the number of significant digits for coordinates part of good GIS design.

Re: The lat/lon floating point delusion

#37

On the flip side - who cares? It's just a question of shunting data around - ok someone could add a format specifier somewhere and it would be neater. But there's no particular harm in it either.

Sending more data than you need is bad for two reasons.

1. It implies more precision, which in many cases is a flat out lie. Savvy users will know to truncate, round or something, but others will be lead astray.

2. It takes more space to represent. If the engineers storing double precious floating points had asked themselves if they needed to be sending these out at such a high resolution, they may have been able to cut the storage of these things down 50%.

Though yes, there are probably bigger issues to worry about. Doesn't mean we should outright dismiss these points.

Re: The lat/lon floating point delusion

#38

Yes, why don't people spend their time doing the extra work to figure out the exact number of meaningful digits in their measurements when the defaults work just fine for their non-scientific, non-metrological purposes? A mystery for the ages.

I like your argument, but it would be nice to keep things civil.

This is the civil version, with mild sarcasm.

Re: The lat/lon floating point delusion

#39
It doesn't matter how precise the value is, as long as it's precise enough for your use-case and as long as machines are handling it. If the value is displayed in a UI and it's rounded off to a reasonable decimal there, I don't see the point why you would get so worked up about it.

I'm sure my thermostat internal temperature representation is way more precise than the 0.5 degrees precision it's showing on the display.

Re: The lat/lon floating point delusion

#40
post #3

Not just lat/lon. I find myself amused when The Washington Post's education columnist Jay Matthews--quite a sharp guy from all I can tell--runs his high school "challenge index" (AP tests taken / size of graduating class) out to six decimal points for small schools. Somebody give that man a slide rule.

I didn't really get significant digits until I played with a slide rule.
Post reply on HN