Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The lat/lon floating point delusion

datafix.com.au

Post reply on HN
Print view View original

91–100 of 154 posts

Re: The lat/lon floating point delusion

#91

Post by Majromax »

whatshisface wrote: post #72
Majromax wrote: post #19

Earlier quoted context omitted.

What numerically unstable algorithms are ever performed with geolocations? Moreover, given the author's point that real measurement errors exceed the false precision of published data, if such a calculation were performed and did provide "arbitrarily large" error, it would indicate that the result should in fact be nonsense.

I'm thinking about the case where a calculation extends across many roundings.

That doesn't necessarily make the algorithm unstable. See for example:

  >>> import numpy as np
  >>> x0 = np.random.rand(1)
  >>> x_rnd = np.round(100*x)/100.0
  >>> for i in range(1000):
  ...  x = np.mod(x+np.pi,1)
  ...  x_rnd = np.mod(x_rnd + np.pi,1)
  ...  x_rnd = np.round(100*x)/100
  ... 
  >>> x_real = np.mod(x0+1000*np.pi,1)
  >>> print(x_real,x,x_rnd)
    (array([0.51013166]), array([0.51013166]), array([0.51]))
Note no loss of accuracy for all of the intermediate roundings. The accuracy is preserved because there's no mechanism here to amplify the error.

That's why I ask about the kind of algorithms typically applied to geolocated data. Off the top of my head, I can't think of anything that would be both useful and error-amplifying.

Re: The lat/lon floating point delusion

#92

Post by falcolas »

7952 wrote: post #82
falcolas wrote: post #80

Earlier quoted context omitted.

They aren't approximations; you can verify this yourself. Pull up Google Maps, pick an arbitrary spot with no businesses, and click. You will get at least 6 significant digit coordinates.

No, I meant that it is an approximation or the thing in the real world that it represents. For example, consider a restraunt. Should the point be at the entrance to the building, in the middle of the building, or maybe at the door round the back where mail is delivered, or at the centroid of the parcel. In order to actually use this excessive precision you would need to know that. In the examle cases the data is not…

Then the coordinates presented should match that approximation. Anyone who has worked with plans knows that the number of significant digits presented on the plans has a meaning; it represents the margin of error.

Online maps aren't respecting this, they're giving decimeter precision values on maps that are lucky if they're accurate to within meters, if not tens of meters. That's the problem.

Re: The lat/lon floating point delusion

#93

Post by cpncrunch »

wpietri wrote: post #79

In contrast to his 4-digits-are-fine notions, let me offer a counterexample: http://scissor.com/transient/destination_heatmap/ I am working on a Twitter bot called @sfships, which monitors the comings and goings of large ships in the San Francisco Bay: https://twitter.com/sfships As part of that, I generated the above map from AIS data. [1] It's basically where ships stop. If you zoom in on the San Francisco waterfro…

Another example: I was looking up my property details yesterday, and wanted to get exact coords for my property line. Their web app lets you click on the map to get coords, but it only displays degrees/minutes/seconds (with no decimal point), so it wasn't much use.

Re: The lat/lon floating point delusion

#94

Post by secabeen »

cafard wrote: post #3

Not just lat/lon. I find myself amused when The Washington Post's education columnist Jay Matthews--quite a sharp guy from all I can tell--runs his high school "challenge index" (AP tests taken / size of graduating class) out to six decimal points for small schools. Somebody give that man a slide rule.

That "index" is terrible. It claims to be a ranking of all schools:

> Schools ranked no. 220 or above are in the top 1 percent of America’s 22,000 high schools, no. 440 or above are in the top 2 percent and so on.

But he only has ~2,000 schools in his database, or less than 10% of all schools.

Re: The lat/lon floating point delusion

#95

Post by logfromblammo »

dfranke wrote: post #75
protonfish wrote: post #65

Earlier quoted context omitted.

The problem is that these lat/lon figures are not displayed with precision information at all, so we can't even know the std dev. Sig figs are a simple and clear way to communicate the precision of a measurement, but if you want to be more statistically accurate, you can use parentheses to indicate the standard deviation. In your example (if I mess this up please correct me, but I think) it would be 45.73490534578(10…

I've never seen the convention you're using and I don't think I understand it. Conventions I've seen include: * Give an error bound like 45.73490534578° (±0.00002°) and indicate in prose that this is a 2σ bound. * Put non-significant figures in parenthesis, like 45.73490(534578)° (EDIT: possibly I've misinterpreted this one when I've seen it, see logfromblammo's reply) * Put a bar over the last significant figure, li…

The value in the parentheses is the symmetric one-sigma bound.

If I say the atomic weight of F is 18.998403163(6) g/mol...

  mean     18.998403163 g/mol
  std.dev.  0.000000006 g/mol
If I say Planck time is 5.391245(60)e−44 s...

  mean     5.391245e−44 s
  std.dev. 0.000060e-44 s
The standard rules for rounding imply that whenever a measurement is given to a certain number of significant figures, you're leaving out "0(5)" from the end. So 1.2345 is 1.23450(5) in parenthetical notation.

Significant figures rules give you a close-enough propagation of error, but in order to be more exact, you need to combine absolute uncertainties when adding or subtracting, and combine relative uncertainties when multiplying or dividing.

Re: The lat/lon floating point delusion

#96

Post by busyant »

dfranke wrote: post #75
protonfish wrote: post #65

Earlier quoted context omitted.

The problem is that these lat/lon figures are not displayed with precision information at all, so we can't even know the std dev. Sig figs are a simple and clear way to communicate the precision of a measurement, but if you want to be more statistically accurate, you can use parentheses to indicate the standard deviation. In your example (if I mess this up please correct me, but I think) it would be 45.73490534578(10…

I've never seen the convention you're using and I don't think I understand it. Conventions I've seen include: * Give an error bound like 45.73490534578° (±0.00002°) and indicate in prose that this is a 2σ bound. * Put non-significant figures in parenthesis, like 45.73490(534578)° (EDIT: possibly I've misinterpreted this one when I've seen it, see logfromblammo's reply) * Put a bar over the last significant figure, li…

You can see it done here: https://en.wikipedia.org/wiki/Standard_atomic_weight#List_of...

Re: The lat/lon floating point delusion

#98

Post by jjoonathan »

falcolas wrote: post #88
jjoonathan wrote: post #85

Earlier quoted context omitted.

If the page in question listed something that could be construed as a property boundary, sure. It doesn't.

Every new home owner gets a "plat" that outlines the boundaries of their property, including GPS coordinates. Using a tool (like Google Maps) to overlay a coordinate with a ground feature isn't exactly absurd.

What does that have to do with the Melbourne Open Data Portal?

If you want to argue that Google Maps could do more to highlight the accuracy/precision limitations of its projected imagery, I'd completely agree -- but that's a different argument.

Re: The lat/lon floating point delusion

#99

Post by gus_massa »

dfranke wrote: post #75
protonfish wrote: post #65

Earlier quoted context omitted.

The problem is that these lat/lon figures are not displayed with precision information at all, so we can't even know the std dev. Sig figs are a simple and clear way to communicate the precision of a measurement, but if you want to be more statistically accurate, you can use parentheses to indicate the standard deviation. In your example (if I mess this up please correct me, but I think) it would be 45.73490534578(10…

I've never seen the convention you're using and I don't think I understand it. Conventions I've seen include: * Give an error bound like 45.73490534578° (±0.00002°) and indicate in prose that this is a 2σ bound. * Put non-significant figures in parenthesis, like 45.73490(534578)° (EDIT: possibly I've misinterpreted this one when I've seen it, see logfromblammo's reply) * Put a bar over the last significant figure, li…

The idea of using something like 45.734905° ± 0.000010° is that the information of the other digits is so small that it's better to ignore it. (Assuming that it is unbiases, and the other digits have some information at all.)

An intuitive way to understand this is considering how much work is necessary to extract information from the next digit. You can use statistic to extract more information. As a rule of thumb you need 100 times more measurements for each additional digit. So you need something like 100 (or 400) for the first digit, 10000 (or 40000) for the second digit and so on. (There is a constant here, I never remember the constant, perhaps it's 4, perhaps it's 1.)

To extract some information from the last 8 in 45.73490534578° you need 1000000000000 measurements! So it's better to just ignore the tiny amount of information in the 8.

In the lab in very controlled scenarios you can repeat a measurement very carefully a lot of times automatically and then use statistics. But if you have a handheld GPS, you can't repeat the measurement more than a few hundred of times.

A device that use a somewhat similar process is the https://en.wikipedia.org/wiki/Lock-in_amplifier it is not exactly this statistic trick, but note that it needs an stable environment to repeat the measurements. Wikipedia says that it can detect a signal 1 million times smaller than noise, but IIRC (the cheap ones?) can only detect a signal that is 1/100 or 1/1000 of the noise..

PS: Please never use 45.73490534578° (±0.00002°). If you use in a lab in the university, the TA will get mad at you. You can try using 45.73490534578° (±0.00002000000°) that is bad but not so horrible bad, the TA will still get mad at you but you may survive.

Re: The lat/lon floating point delusion

#100

Post by krapht »

caf wrote: post #32
Luc wrote: post #18

Earlier quoted context omitted.

Which calculation on lat/lon coordinates would make an atom-scale error grow, say, a million times to millimeter-size?

"Build the gravitational wave observatory between the Thai restaurant and the railway station"

You joke but a lot of radar/radio observation is done with ECEF coordinates, which is a (lat, lon, z-axis) triple.
Post reply on HN