Earlier quoted context omitted.
> That way we can evaluate after the fact how well people did. How exactly do you propose to do that given the binary knowledge of the event occuring or not...
Let's say somebody makes a prediction that something will happen, and it doesn't. You could evaluate the prediction differently if they gave it a 55% chance of happening vs a 95% chance.
But what if I predict with 100% certainty heads, and the coin comes up heads. Did we learn anything "useful" about my prediction?