Via Turing, we know that the Predictor would be able to decide programs, and since it can't, it can't be a Predictor.
The isomorphism between the program which defeats decidability and the two-box Newcomb problem is left as an exercise.
Newcomb's paradox
21–30 of 70 posts
Re: Newcomb's paradox
#22I always find this kind of philosophical thought experiment unsatisfying. Super-accurate predictions of human behaviour are just not possible. If I could do it, I'd be a gazillionaire philanthropist/playboy dating supermodels and advising heads of state because I can't be bothered to rule the world directly. As it is, I can't do better than a draw against a 5-year-old at rock-paper-scissors. So this paradox tells us…
I can be more interesting reframed as such:
The player is the AI, and the predictor is the AI programmer.
The AI programmer can look into the innards of the AI ie. the source code and can thus predict with high accuracy what the AI will do.
What is a winning strategy for the AI?
Or taken another way, you can have AI's that have access to each other's source code and are competing for some scarce resource, how do you design an AI that 'wins' when it's behaviour is known it's opponent?
Re: Newcomb's paradox
#23Not sure why it is a paradox - assuming the predictor is superintelligent, you don't try to fool it. By definition its intelligence can predict what you will do in the very last moment, so the fact that it doesn't get to change anything once prediction is made, is immaterial.
Let's reverse the numbers and make both boxes transparent.
If The Predictor suspects you will choose just box A, it'll put 1 million in box B and 1 thousand in box A, if it suspects you will take both A and B, it will put nothing anything in box B and 1 thousand in box A.
So now you standing in front of the two transparent boxes. You see that there is $1 million in box B, yet you still take just box A?
Re: Newcomb's paradox
#24I always find this kind of philosophical thought experiment unsatisfying. Super-accurate predictions of human behaviour are just not possible. If I could do it, I'd be a gazillionaire philanthropist/playboy dating supermodels and advising heads of state because I can't be bothered to rule the world directly. As it is, I can't do better than a draw against a 5-year-old at rock-paper-scissors. So this paradox tells us…
I can be more interesting reframed as such: The player is the AI, and the predictor is the AI programmer. The AI programmer can look into the innards of the AI ie. the source code and can thus predict with high accuracy what the AI will do. What is a winning strategy for the AI? Or taken another way, you can have AI's that have access to each other's source code and are competing for some scarce resource, how do you…
Sure you can. The AIs behaviour could be as simple as "Always pick box B".
The difficult bit would be designing an AI which, given perfect knowledge of its logic, would pick both boxes despite appearing to be more likely to pick only B.
In that case, you could simply have an AI with a 0.499999 chance of picking both and a 0.5000001 chance of picking B. The expected winnings would be $1,000,500.
But then, once it comes down to probability, the predictor is no longer a 'perfect predictor' any more.
Re: Newcomb's paradox
#25Not sure why it is a paradox - assuming the predictor is superintelligent, you don't try to fool it. By definition its intelligence can predict what you will do in the very last moment, so the fact that it doesn't get to change anything once prediction is made, is immaterial.
Let's reverse the numbers and make both boxes transparent. If The Predictor suspects you will choose just box A, it'll put 1 million in box B and 1 thousand in box A, if it suspects you will take both A and B, it will put nothing anything in box B and 1 thousand in box A. So now you standing in front of the two transparent boxes. You see that there is $1 million in box B, yet you still take just box A?
In that scenario, it wouldn't make a difference what you choose.
If you were so inclined to take Box B's million, Omega would never have put that million in the first place.
The only way for Omega to put that million there is if you weren't inclined to take Box B - despite the million being completely visible to you - in which case you lose anyway.
The situation where Omega gives you the million, and you take it, just never comes up. Can't fool it.
Re: Newcomb's paradox
#26Via Turing, we know that the Predictor would be able to decide programs, and since it can't, it can't be a Predictor. The isomorphism between the program which defeats decidability and the two-box Newcomb problem is left as an exercise.
I think this may be the proof.
Start by assuming that the predictor is a decider P for a computational model of a human M in a situation encoded by w. We can then construct a Turing Machine S that decides A_TM.
S {
1. Run P on w.
2. If P halts, run M on w; if M accepts, accept, if it rejects reject.
}
Since S decides A_TM we must have made a mistake somewhere in our proof, our only mistake could be that the predictor P is a decider. Therefore the Predictor is not a decider.
Re: Newcomb's paradox
#27I always find this kind of philosophical thought experiment unsatisfying. Super-accurate predictions of human behaviour are just not possible. If I could do it, I'd be a gazillionaire philanthropist/playboy dating supermodels and advising heads of state because I can't be bothered to rule the world directly. As it is, I can't do better than a draw against a 5-year-old at rock-paper-scissors. So this paradox tells us…
I don't think the schism is because of fallible vs infallible so much as people being stuck in naive decision theory (just take both boxes; it's already decided beforehand!) or not (hey, the kinds of decisions I'm willing to make might have changed what Omega decided in the past!).
After all, Omega need not be infallible. So long as he predicts your decision with an accuracy of 50.05% (slightly better than a coin toss), you profit:
---
Let p be the probability that Omega predicts your decision correctly.
E(one-boxing) = p⋅$1mil + (1-p)⋅0
E(two-boxing) = (1-p)⋅$1.01mil + p⋅$1k
Solving for E(one-boxing) > E(two-boxing)
p⋅$1mil > (1-p)⋅$1.01mil + p⋅$1k
p($1mil + $1.01mil - $1k) > $1.01mil
p⋅2mil > $1.01mil
p > 50.05%
---
And if he doesn't predict you slightly better than a coin toss, why is he called the Predictor?
Re: Newcomb's paradox
#28I always find this kind of philosophical thought experiment unsatisfying. Super-accurate predictions of human behaviour are just not possible. If I could do it, I'd be a gazillionaire philanthropist/playboy dating supermodels and advising heads of state because I can't be bothered to rule the world directly. As it is, I can't do better than a draw against a 5-year-old at rock-paper-scissors. So this paradox tells us…
It's unsatisfying because it's so poorly defined. As you attempt to define it more precisely, the problem just converges on the problem of whether free will exists.
Re: Newcomb's paradox
#29For the risk-averse: Make a bet with someone for $500,000.00 that you can prove The Predictor is fallible. Take only box B. If box B contains $1,000,000.00, then you have lost the bet and you are left with only $500,000.00. If box B contains no money, you have wone the bet and are left with $500,000.00.
Who would take the other side of that bet?
Re: Newcomb's paradox
#30I think there's a weak echo of this concept that plays out in an election. Except swap in the inconvenience of voting for giving up the $1,000 box. On the one hand, why bother voting? It's a pain in the ass and my single vote has such a negligible effect. On the other hand, if everyone like me has that same attitude, then I and others like me lose our voice in the election. So should I vote, or not?
That's a fairly simple one. If all you value is the benefits to you of your influence on the election, do not bother to vote (and especially don't bother spending the resources required to educate yourselves so as to vote responsibly). It's very clearly not worth it except perhaps for the smallest local elections or closest large elections.
If you value other things related to voting, like feeling as if you've done your civic duty, or feeling like a member of a group (like a political party), then by all means vote if the hassle of physically voting is less than that benefit.
This seems like a pretty clear description of rational behavior, and if you expect people to behave largely rationally, this description seems to explain some of the problems often attributed to elections, like low voter participation and low voter knowledge of the candidates and issues.