31–40 of 48 posts
Earlier quoted context omitted.
Mathematician here. You can't have a uniform distribution on a set with infinite measure. Unbounded sets are fine as long as they have finite measure.
What is an unbounded set with finite measure?
Earlier quoted context omitted.
Mathematician here. You can't have a uniform distribution on a set with infinite measure. Unbounded sets are fine as long as they have finite measure.
What is an unbounded set with finite measure?
The concepts are a bit sloppy here. "Zero information" is a bit of hand waving which relies on the idea of having a uniform distribution on a set with infinite measure, which is not mathematically sound nor is it useful as an approximation. All messages that you choose to transmit follow some probability distribution with each message having finite, nonzero probability. This is what the author gets wrong, on a very f…
Hash functions. In case of a good hash function the information about the actual distribution of the inputs is erased. That the hashes are uniformly distributed does not imply that the inputs are uniformly distributed which in turn renders any conclusion about the distribution of the inputs after applying a transformation invalid.
Uninformative priors. This seems to be more of a philosophical problem I don't really know much about. But as far as I can tell if one tries to quantify a lack of information in a naive way using probability distributions, then one gains information, for example that the value is uniformly distributed over some range, and this information has consequences like specific distributions of derived values.
So the attempt to quantify a lack of information turns into a self-defeating endeavor, not necessarily because of any inherent information but because of the information injected during the modeling process.
If we have any probability distribution, that tells us something about x. If someone tells us that license plates numbers are uniformly distributed, we can pretend to be a license plate maker by sampling from a uniform distribution, and nobody else could tell the difference by looking at the license plate numbers we make. Zero information is more like not knowing what probability distribution a variable comes from. R…
Uniform distribution gives minimum information, assuming the range is known. Any other distribution will give more information. Talking about information capacity of coded messages, here. I think what you are saying when you use the phrase "zero information" is really "zero knowledge".
The article seems to want to say that "zero information" ought to be a probability distribution such that for all functions f, f(X) is the same zero-information distribution, i.e. from nothing we get nothing. The point is that no such X exists, because every probability distribution encodes some amount of information. What we want is some function F such that for every X in a large class of probability distributions, F(X) is uniform. Which is exactly why x^2 is a terrible hash function.
As far as I know, there's no publicly known one-way cryptographic function for which there's a provable minimum level of effort to invert.
On the hash function front, where the author seems concerned about something, perhaps the lesson is that filling up hash tables to the 90% level before expansion is pushing too hard.[1] At 70% fill, the hash function doesn't have to be near-perfect, just not awful.
[1] http://accidentallyquadratic.tumblr.com/post/153545455987/ru...
I think information can not be defined based on the source material alone. Why don't we define information like this? An observer X has a prior estimate of the distribution d1 of variable x. The source material yields a distribution d2 of variable x. Then for that observer, the information gained from the source is something like |d1-d2|.
The ideal hash function leaks no information. In practice, we can only make one leak very little over a long time.
The author considers speed of light being the absolute, irrevocable certainty, but how does he know that? Does he know _why_ c is constant? Had he mastered general relativity enough to be 100% sure that it can't vary in time or space? Or that our current understanding of physics is total and there absolutely can't be something that we don't know about speed of light? https://en.m.wikipedia.org/wiki/Time-variation_of_…
Earlier quoted context omitted.
Mathematician here. You can't have a uniform distribution on a set with infinite measure. Unbounded sets are fine as long as they have finite measure.
A point distribution (single point) might be considered uniformly distributed. :-)
The author considers speed of light being the absolute, irrevocable certainty, but how does he know that? Does he know _why_ c is constant? Had he mastered general relativity enough to be 100% sure that it can't vary in time or space? Or that our current understanding of physics is total and there absolutely can't be something that we don't know about speed of light? https://en.m.wikipedia.org/wiki/Time-variation_of_…
They know it because it's the best theory we currently have. Things aren't 100% sure in physics, there are no laws. Just the best available theory which you should treat as fact while knowing it may not be.