Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Exam Credit for Knowing What You Do Not Know

daviddfriedman.blogspot.com

Post reply on HN
Print view View original

41–50 of 51 posts

Re: Exam Credit for Knowing What You Do Not Know

#41

Post by jumpCastle »

derefr wrote: post #33
Cyph0n wrote: post #27

Earlier quoted context omitted.

Why would anyone ever put 0% then? I would just redefine "impossible" to be an arbitrary percentage that guarantees I don't nuke the question. Quick question: is confidence taken in percent form (ln(90)) or decimal form (ln(0.9))? Edit: I am aware of the implication of having percent values, guys/gals. But if it's in decimal form, a 5% confidence level will end up losing you the current question AND three others.

I assume that was part of the lesson: in Bayesian decision theory, an agent that assigns 0.0 or 1.0 confidence to something can no longer update on new evidence about that thing. Saying something is 0% likely means being so sure about that thing, that nothing—no matter how amazing—could or would ever change your mind. Basically, no human is actually ever 0.0 or 1.0 sure of anything . Assign five or ten nines of confi…

But then what is my confidence that God exists? I would say zero because of the spaghetti monster argument

Re: Exam Credit for Knowing What You Do Not Know

#42

Post by hyperpape »

CaseyJParker wrote: post #39
hectormalot wrote: post #38

Earlier quoted context omitted.

I had a professor that used a slightly different approach. A correct answer would yield you 1pt, but any of the 3 incorrect answers would subtract 0.5pts. Expected average result for guessing is negative, so he incentivized people to leave questions open if they were unsure.

The expected average for a straight guess with absolutely zero exposure to the material would still be slightly positive. If nothing else, there's intuition and ingenuity, which place a person just a touch above a truly random guess-generating-machine.

With four answers, you get .25 + (.75)(-.5) = -.125 for guessing.

Edit: yeah, I badly misread this.

Re: Exam Credit for Knowing What You Do Not Know

#43

Post by hyperpape »

derefr wrote: post #33
Cyph0n wrote: post #27

Earlier quoted context omitted.

Why would anyone ever put 0% then? I would just redefine "impossible" to be an arbitrary percentage that guarantees I don't nuke the question. Quick question: is confidence taken in percent form (ln(90)) or decimal form (ln(0.9))? Edit: I am aware of the implication of having percent values, guys/gals. But if it's in decimal form, a 5% confidence level will end up losing you the current question AND three others.

I assume that was part of the lesson: in Bayesian decision theory, an agent that assigns 0.0 or 1.0 confidence to something can no longer update on new evidence about that thing. Saying something is 0% likely means being so sure about that thing, that nothing—no matter how amazing—could or would ever change your mind. Basically, no human is actually ever 0.0 or 1.0 sure of anything . Assign five or ten nines of confi…

I suppose this is a bad place to ask this, but given that you need to do math to update your credences and also that you can derive any proposition from Pr('2 + 2 = 4) != 1, how does this work?

Re: Exam Credit for Knowing What You Do Not Know

#44

Post by Cyph0n »

jumpCastle wrote: post #41
derefr wrote: post #33

Earlier quoted context omitted.

I assume that was part of the lesson: in Bayesian decision theory, an agent that assigns 0.0 or 1.0 confidence to something can no longer update on new evidence about that thing. Saying something is 0% likely means being so sure about that thing, that nothing—no matter how amazing—could or would ever change your mind. Basically, no human is actually ever 0.0 or 1.0 sure of anything . Assign five or ten nines of confi…

But then what is my confidence that God exists? I would say zero because of the spaghetti monster argument

He's right though, you can never be truly certain that some kind of God doesn't exist. But if you're referring to a specific God, then maybe.

Re: Exam Credit for Knowing What You Do Not Know

#45

Post by cercatrova »

hyperpape wrote: post #42
CaseyJParker wrote: post #39

Earlier quoted context omitted.

The expected average for a straight guess with absolutely zero exposure to the material would still be slightly positive. If nothing else, there's intuition and ingenuity, which place a person just a touch above a truly random guess-generating-machine.

With four answers, you get .25 + (.75)(-.5) = -.125 for guessing. Edit: yeah, I badly misread this.

That's what the parent is saying to be not applicable to the situation; humans are not simply true random guessing machines but are able to reason and eliminate choices, which would increase the expected value to be positive.

Re: Exam Credit for Knowing What You Do Not Know

#46

Post by derefr »

hyperpape wrote: post #43
derefr wrote: post #33

Earlier quoted context omitted.

I assume that was part of the lesson: in Bayesian decision theory, an agent that assigns 0.0 or 1.0 confidence to something can no longer update on new evidence about that thing. Saying something is 0% likely means being so sure about that thing, that nothing—no matter how amazing—could or would ever change your mind. Basically, no human is actually ever 0.0 or 1.0 sure of anything . Assign five or ten nines of confi…

I suppose this is a bad place to ask this, but given that you need to do math to update your credences and also that you can derive any proposition from Pr('2 + 2 = 4) != 1, how does this work?

Basically: probabilities and credences aren't the same thing. They're measured on the same scale, but probabilities can be used to reason about either facts or evidence, while credences are only "about" evidence (though potentially evidence about facts.)

To put that another way: it is "100% true" that 2 + 2 = 4 (in ZF set theory); just like it is "100% true" that the sun will rise today, because it already did (and otherwise the Monty Hall problem would have a different answer.)

But the credence in an agent's mind can and should only be informed by observations of these facts; and the "observing hardware" cannot be 1.0 trustworthy. (Your senses can be imprecise; the brain can be buggy or biased; the environment being observed can be the construction of a Cartesian "evil demon"; etc.)

Given an artificial agent with perfect knowledge (e.g. one who is able to directly entangle itself with the environment it is trying to predict, rather than just one able to entangle itself with evidence about the state of an environment), some things can theoretically be held to have 0.0 or 1.0 probabilities (e.g. events that already happened)—but given an embodied agent, the "maybe my processing is flawed" argument should still make these into non-infinite credences.

Re: Exam Credit for Knowing What You Do Not Know

#47

Post by hyperpape »

derefr wrote: post #46
hyperpape wrote: post #43

Earlier quoted context omitted.

I suppose this is a bad place to ask this, but given that you need to do math to update your credences and also that you can derive any proposition from Pr('2 + 2 = 4) != 1, how does this work?

Basically: probabilities and credences aren't the same thing. They're measured on the same scale , but probabilities can be used to reason about either facts or evidence, while credences are only "about" evidence (though potentially evidence about facts.) To put that another way: it is "100% true" that 2 + 2 = 4 (in ZF set theory); just like it is "100% true" that the sun will rise today, because it already did (and…

Thanks. I should've remembered that the connection to an agent's credences and their judgments of probabilities is complicated.

Re: Exam Credit for Knowing What You Do Not Know

#48

Post by platz »

derefr wrote: post #46
hyperpape wrote: post #43

Earlier quoted context omitted.

I suppose this is a bad place to ask this, but given that you need to do math to update your credences and also that you can derive any proposition from Pr('2 + 2 = 4) != 1, how does this work?

Basically: probabilities and credences aren't the same thing. They're measured on the same scale , but probabilities can be used to reason about either facts or evidence, while credences are only "about" evidence (though potentially evidence about facts.) To put that another way: it is "100% true" that 2 + 2 = 4 (in ZF set theory); just like it is "100% true" that the sun will rise today, because it already did (and…

If the Higgs field quantum tunnels to a lower energy level today in our vincinity, the sun will not rise tomorrow. Never put 100%

Re: Exam Credit for Knowing What You Do Not Know

#49

Post by derefr »

platz wrote: post #48
derefr wrote: post #46

Earlier quoted context omitted.

Basically: probabilities and credences aren't the same thing. They're measured on the same scale , but probabilities can be used to reason about either facts or evidence, while credences are only "about" evidence (though potentially evidence about facts.) To put that another way: it is "100% true" that 2 + 2 = 4 (in ZF set theory); just like it is "100% true" that the sun will rise today, because it already did (and…

If the Higgs field quantum tunnels to a lower energy level today in our vincinity, the sun will not rise tomorrow. Never put 100%

I didn't say "the sun will rise tomorrow"; I said "the sun will rise today"—as in, the (trivial) probability that the sunrise that already happened, will happen, looking forward from the more distant past to the more recent past. Given that we are talking about the probability of that event occurring from a reality where it occurred, the updated probability of it occurring is 100%. (But we cannot have 100% credence, simply because we can't actually be 100% sure we're in said reality.)

Re: Exam Credit for Knowing What You Do Not Know

#50

Post by gpawl »

_yosefk wrote: post #36
mathattack wrote: post #30

Side note - if the article sounds uber-rational econ, it may be in part because the author is the son of Milton Friedman. Here is his first post on the topic. http://daviddfriedman.blogspot.com/2012/11/the-use-of-old-ex... There is a great quote at the end. And, for a last comment ... . I like to say that being a professor is better than working for a living, except when grading exams. One reason is that grading exam…

While it's hard to argue that it runs in the family to some extent in this case, David Friedman is an econ thinker in his own right and certainly goes way further into what most would call extreme libertarian/"uber-rational econ" territory than Milton Friedman ever did. In fact, I'd say that "The Machinery of Freedom" ought to entitle David Friedman to the simpler introduction "...because the author is David Friedman…

His most famous book his named after his father's most famous book. His father clearly sets the context for his work.
Post reply on HN