Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is Probability Real?

arameb.com

Post reply on HN
Print view View original

41–50 of 227 posts

Re: Is Probability Real?

#41

Post by e12e »

st1x7 wrote: post #8
enriquto wrote: post #5

Earlier quoted context omitted.

A simple sentence that I've found useful for pedagogy: "the probability of that coin toss being 50% does not talk about the coin; it talks about you , and about your partial knowledge of the universe." You can add: "The coin toss itself is deterministic and the result can be computed if you know the initial position and speed." They will inevitably bother you about the physical impossibility to measure the starting p…

> the probability of that coin toss being 50% does not talk about the coin; it talks about you, and about your partial knowledge of the universe. But it does talk about the coin - a weighted coin would have a different probability. Same in the example with the white/black balls - if they weren't 5 white and 5 black but 6 white and 4 black, the probability you would assign to the top one would be different. Again, the…

I like the balls in container better - the mechanics of determinism is more obvious - and the contrast between perfect and partial knowledge.

And there's another interesting point - you could view "there's five white and five black balls" as your model. If in reality, there's one white ball and nine black - then your math is still right, but your model is wrong.

If you do experiments with the wrong model (assuming 50/50, getting samples from 1/10) - your best conclusion would be the model is wrong.

But for many settings, you'd end up declaring your container or the hand used to pull out balls has magical powers. (and to borrow from Douglas Adams go on to prove that black is white, and get killed in the next zebra crossing).

Re: Is Probability Real?

#42

Post by smallnamespace »

MereInterest wrote: post #39
enriquto wrote: post #5

Earlier quoted context omitted.

A simple sentence that I've found useful for pedagogy: "the probability of that coin toss being 50% does not talk about the coin; it talks about you , and about your partial knowledge of the universe." You can add: "The coin toss itself is deterministic and the result can be computed if you know the initial position and speed." They will inevitably bother you about the physical impossibility to measure the starting p…

As a caveat, while this intuition works for classical mechanics, it does not work for quantum mechanics. All observations are consistent with wave function collapse being fundamentally random. Any hidden variables would need to be transmitted many times faster than the speed of light (~10000x, last time I checked the experiments), and are therefore inconsistent with our understanding of special relativity.

Note that pilot wave theory, an (out of vogue) interpretation of quantum mechanics, also recasts the apparent randomness in quantum mechanics as due to our ignorance of the exact state of the pilot wave.

Even Einstein struggled with quantum mechanics, famously saying "[God] does not play dice with the universe".

Re: Is Probability Real?

#43

Post by sn41 »

qsort wrote: post #33
SmooL wrote: post #23

Earlier quoted context omitted.

Isn't that what the occam's razor argument was for? Sure, both an RNG and the "40 digits of pie" program can produce that output, but the "40 digits of pie" program is shorter, therefore having less kolmogorov complexity

Yeah but is it? They are both programs with logarithmic Kolmogorov complexity, and Kolmogorov complexity is only defined up to constant factors unless you commit to a computational model. If you do commit to one, which is the shortest is just a function of what are the specifics of the model you chose, which isn't really interesting, it's literally code golf at that point.

In order to discriminate between the output of an RNG and the digits of PI, you are right that they have the same Kolmogorov Complexity to within additive constants.

However, you can ask for resource-bounded Kolmogorov complexity. Suppose you ask for the shortest length of the description of a PTIME machine which output the N bits of an RNG versus that which outputs the first N bits of PI. Complexity theorists believe that the first will be longer. Proving that conclusively might possibly have a bearing on P=BPP.

When you go all the way down to finite-state machines, KC becomes something equivalent to finite-state information-lossless compression, like Lempel-Ziv compressibility.

Long story short: by restricting the resources available for the computation, it is possible to discriminate among some such examples as you point out.

Re: Is Probability Real?

#44

Post by jvanderbot »

st1x7 wrote: post #8
enriquto wrote: post #5

Earlier quoted context omitted.

A simple sentence that I've found useful for pedagogy: "the probability of that coin toss being 50% does not talk about the coin; it talks about you , and about your partial knowledge of the universe." You can add: "The coin toss itself is deterministic and the result can be computed if you know the initial position and speed." They will inevitably bother you about the physical impossibility to measure the starting p…

> the probability of that coin toss being 50% does not talk about the coin; it talks about you, and about your partial knowledge of the universe. But it does talk about the coin - a weighted coin would have a different probability. Same in the example with the white/black balls - if they weren't 5 white and 5 black but 6 white and 4 black, the probability you would assign to the top one would be different. Again, the…

> But it does talk about the coin - a weighted coin would have a different probability.

Nobody would a priori assume it is a weighted coin, in this example. A coin is chosen because it's been a standard weight and measure-backed object for centuries. It is about the observer's knowledge, which includes assumptions from every day experience.

You have to base a prior on assumptions. If you assume nothing and flip it 1000x, and calculate the probability, than you base that on assumptions of your own flipping ability, and hand wave it away with count divided by trials.

Re: Is Probability Real?

#45

Post by shawnz »

st1x7 wrote: post #13
wearsshoes wrote: post #12

Earlier quoted context omitted.

This is still a partial knowledge situation - you have the information that there are a certain proportion of colored balls in the chamber, but not the information about their order. The probability includes the information we do know and allows inferences about information we don’t know.

Sure, I just can't agree with the parent statement that probability has nothing to do with the object it describes, which is demonstrably false.

They didn't say that it has "nothing to do with the object it describes" though. It has to do with your knowledge, which itself has to do with the object your knowledge describes

Re: Is Probability Real?

#46

Post by posterboy »

SmooL wrote: post #23
qsort wrote: post #21

Earlier quoted context omitted.

The article is excellent, congratulations. A couple observations/questions. 1) You didn't comment on the bayesian viewpoint that probability reflects a subjective idea about the state of the world. One might argue, for example, that probability isn't measurable, and that therefore, strictly speaking, a statement about the objective probability of an event isn't meaningful. Experimental evaluation would have to be don…

Isn't that what the occam's razor argument was for? Sure, both an RNG and the "40 digits of pie" program can produce that output, but the "40 digits of pie" program is shorter, therefore having less kolmogorov complexity

Is it really shorter?

For all I know this is a false dichotomy. True RNGs don't exist, for one, in a deterministic philosophy.

RNGs are per definition non-deterministic, but algorithms are deterministic--by the definition that I learned. We use floating gates picking up cosmic background radiation or the fallout from radioactive elements to come close to really random. XKCD's butterfly joke applies.

Computable numbers have a computable complexity. For a non-deterministic program the fair comparison to a true RNG would be a programm that outputs all digits of Pi up to infinity, I suppose. Vice-versa, I'm not sure if a pseudo-random number generator couldn't be fit into the equivalent kolmogorov complexity of a 40-digits of PIE.

Are you saying it couldn't be? On the one-hand it is the case that pseudo RNGs simply rely on intractible complexity, which must regularly supercede 40 digits of Pi equivalent to 40 byte key, as opposed to your 4kb RSA key chains and what not.

On the other hand I think your argument is akin to the gamblers fallacy: I have absolutely no clue but I will hope it will not have been a primitive RNG, right?

For reference, here's a simple pseudo RNG taken from TinyPTC example code to produce a TV-noise graphic in a loop

        noise = seed;
        noise >>= 3;
        noise ^= seed;
        carry = noise & 1;
        noise >>= 1;
        seed >>= 1;
        seed |= (carry

Re: Is Probability Real?

#47

Post by smitty1e »

> “Given competing programs whose outputs match our observations, always prefer the shortest.”

I need to know context. Output is necessary, but not sufficient. Is this a stand-alone chat server for some niche voices, or, say, Twitter?

Re: Is Probability Real?

#48

Post by willis936 »

MauranKilom wrote: post #11

> The scientific method only works because the rules of the universe happen to be simple, while the set of observations it offers is vast. Kolmogorov complexity captures this defining characteristic of our reality. I've never seen this spelled out so beautifully!

Not everything that is true is simple or catchy. Biasing towards it is a trap.

PBS SpaceTime has a good discussion on this topic.

https://youtu.be/xFKgIOX8IRE

Re: Is Probability Real?

#49

Post by loup-vaillant »

Tow things.

First, probability is real: it's a construct in one's mind, and minds are just as real as dice or coins. https://www.lesswrong.com/posts/f6ZLxEWaankRZ2Crv/probabilit...

Second, (and the author may be leading up to this), there's Solomonoff's theory of inductive inference, which he has proven complete: when we apply Occam's razor (where the prior probability of each possible theory drops exponentially with its size), the amount of error a perfect Bayesian makes as they observe event and bet on the next one, ad infinitum, is finite. Roughly proportional to the complexity of the simplest theory that correctly predict the whole sequence of events. It's one of the most convincing proofs that Bayesian reasoning works. https://en.wikipedia.org/wiki/Solomonoff's_theory_of_inducti...

There's just a little snag. Perfect Bayesian reasoning is impossible to compute, so us mortals have to resort to approximations. Just as perfect certainty isn't possible, perfect reasoning is not attainable. Oh well.

Re: Is Probability Real?

#50

Post by JamisonM »

EbTech wrote: post #2

Since this topic isn't so well-known, I wrote the case arguing that frequentist interpretations don't work, but algorithmic information theory (Kolmogorov complexity) does. I want to make this accessible and persuasive, so thoughts, questions, and arguments would be appreciated!

Some thoughts on the composition:

* "I wrote the case arguing that frequentist interpretations don't work, but algorithmic information theory does": if that's what you are up to here then I think it would be for readers if you stated that up front in some way. And hit me with some kind of summary at the end that makes the concise version of your argument at the end, it's a long article.

* Shorter might be better: There's a lot of stuff in here that I think you can pare out in the probability discussion that maybe isn't adding that much to your argument. I think there is a lot to be gained by assuming a generous reader.

* Betting might be a distraction to your point: This might be confusing the imperfect knowledge of participants in a market with the imperfect knowledge of all the physical forces involved in a physical phenomenon and how that related to the seeming "randomness" of a coin flip for your reader. (The liquidity and stuff.. this is just not related to your point.)

* Don't undermine your point with unrelated assumptions: "I imagine they wouldn’t consider their world unlikely at all: they would just add a new law to their description of physics: all dice, as if by divine intervention, are deemed to exhibit this strange behaviour" this lead me to think that you were just sort of shooing away the whole last X decades of high vs. low energy physics, we collectively certainly don't think that we have the rules correct precisely because of this complication, we find the idea that we need 2 sets of rules improbable and believe that there must be a way to explain everything with a single set of rules. So your mythical dice society probably would consider their dice exception a very unlikely world.. they would be confident they have the world wrong!

A dubious assertion (or at least one that would need a whole lot of explanation) can be an off-ramp for a subset of readers.

Post reply on HN