Exam Credit for Knowing What You Do Not Know • Hacker News Board

Re: Exam Credit for Knowing What You Do Not Know

#21

Post by icebraining » Sat, Mar 04, 2017, 8:57 PM UTC

OisinMoran wrote: post #9

This is a great idea! I think there should definitely be more emphasis placed on knowing what you don't know in a good education system. Even something like knowing what areas of study exist can be quite valuable. I had a friend acting as a research assistant to a biology masters student who was tasked with removing duplicates from an Excel file. The masters student said it would take about four hours because that's…

That sounds a little uncharitable to the student. I know that cars are good for carrying stuff, yet I don't rent one to carry my groceries for three miles from the store to my home. Finding a programmer willing to do it for free might alone take her more than four hours.

Re: Exam Credit for Knowing What You Do Not Know

#22

Post by CuriousCosmic » Sat, Mar 04, 2017, 9:03 PM UTC

whack wrote: post #18

There was a Decision Analysis class at Stanford that took this idea to an extreme. For each multiple choice option, you had to indicate the percentage confidence of that option being correct. The percentages for all 4 options had to add up to 100%. The score you got for each question = 1 + ln(confidence-assigned-for-correct-answer). So if you were completely sure and gave the correct answer 100%, you would get a scor…

I have heard of this before and while it horrifies me as a student, I would love to take a class with a policy like this.

Re: Exam Credit for Knowing What You Do Not Know

#23

Post by TillE » Sat, Mar 04, 2017, 9:05 PM UTC

pirocks wrote: post #11

This is already done by certain tests like the AMC and the old SAT.

This used to be the case on AP exams too, prior to 2011 according to Google. 0 points for a blank answer, but -0.25 for a wrong answer.

Re: Exam Credit for Knowing What You Do Not Know

#24

Post by OisinMoran » Sat, Mar 04, 2017, 9:07 PM UTC

icebraining wrote: post #21

OisinMoran wrote: post #9

This is a great idea! I think there should definitely be more emphasis placed on knowing what you don't know in a good education system. Even something like knowing what areas of study exist can be quite valuable. I had a friend acting as a research assistant to a biology masters student who was tasked with removing duplicates from an Excel file. The masters student said it would take about four hours because that's…

That sounds a little uncharitable to the student. I know that cars are good for carrying stuff, yet I don't rent one to carry my groceries for three miles from the store to my home. Finding a programmer willing to do it for free might alone take her more than four hours.

Sorry I probably should have emphasized that this wasn't a once off task but was very frequent.

Re: Exam Credit for Knowing What You Do Not Know

#25

Post by mdholloway » Sat, Mar 04, 2017, 9:16 PM UTC

I was hoping this would look something more like, given a scenario and a question about it, being able to clearly identify the pieces of information without which the question is unanswerable.

That said, time saving and BS reduction are also valuable goals!

Re: Exam Credit for Knowing What You Do Not Know

#26

Post by nickkoul » Sat, Mar 04, 2017, 9:20 PM UTC

whack wrote: post #18

There was a Decision Analysis class at Stanford that took this idea to an extreme. For each multiple choice option, you had to indicate the percentage confidence of that option being correct. The percentages for all 4 options had to add up to 100%. The score you got for each question = 1 + ln(confidence-assigned-for-correct-answer). So if you were completely sure and gave the correct answer 100%, you would get a scor…

In reality even in your most confident answers couldn't you put 97% on the one you believe is correct and 1% on all others ensuring that you will not receive an ln(0).

Re: Exam Credit for Knowing What You Do Not Know

#27

Post by Cyph0n » Sat, Mar 04, 2017, 9:21 PM UTC

whack wrote: post #18

There was a Decision Analysis class at Stanford that took this idea to an extreme. For each multiple choice option, you had to indicate the percentage confidence of that option being correct. The percentages for all 4 options had to add up to 100%. The score you got for each question = 1 + ln(confidence-assigned-for-correct-answer). So if you were completely sure and gave the correct answer 100%, you would get a scor…

Why would anyone ever put 0% then? I would just redefine "impossible" to be an arbitrary percentage that guarantees I don't nuke the question.

Quick question: is confidence taken in percent form (ln(90)) or decimal form (ln(0.9))?

Edit: I am aware of the implication of having percent values, guys/gals. But if it's in decimal form, a 5% confidence level will end up losing you the current question AND three others.

Re: Exam Credit for Knowing What You Do Not Know

#28

Post by Cyph0n » Sat, Mar 04, 2017, 9:22 PM UTC

nickkoul wrote: post #26

whack wrote: post #18

There was a Decision Analysis class at Stanford that took this idea to an extreme. For each multiple choice option, you had to indicate the percentage confidence of that option being correct. The percentages for all 4 options had to add up to 100%. The score you got for each question = 1 + ln(confidence-assigned-for-correct-answer). So if you were completely sure and gave the correct answer 100%, you would get a scor…

In reality even in your most confident answers couldn't you put 97% on the one you believe is correct and 1% on all others ensuring that you will not receive an ln(0).

It actually depends on how they take the confidence value. If it's percentage, I agree with you. But if it's in numerical form - i.e., 0.01 - you'd still lose that question and a few others.

Re: Exam Credit for Knowing What You Do Not Know

#29

Post by redsn0w422 » Sat, Mar 04, 2017, 9:27 PM UTC

At UIUC, Professor Erickson has a policy that rewards students with 25% credit for indicating they don't know the answer to a particular question, both on homework problems and exams.

[1] https://courses.engr.illinois.edu/cs473/sp2017/exam-policy.h...

Re: Exam Credit for Knowing What You Do Not Know

#30

Post by mathattack » Sat, Mar 04, 2017, 9:55 PM UTC

Side note - if the article sounds uber-rational econ, it may be in part because the author is the son of Milton Friedman. Here is his first post on the topic. http://daviddfriedman.blogspot.com/2012/11/the-use-of-old-ex... There is a great quote at the end.

And, for a last comment ... . I like to say that being a professor is better than working for a living, except when grading exams. One reason is that grading exams is a pain. Another is that it is when you find out that you have not done nearly as good a job of teaching as you thought you had.