Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fighting Fire with Fire: Scalable Oral Exams

behind-the-enemy-lines.com

Post reply on HN
Print view View original

31–40 of 288 posts

Re: Fighting Fire with Fire: Scalable Oral Exams

#31

Post by Wowfunhappy »

ted_dunning wrote: post #22
Wowfunhappy wrote: post #8

...if I was a student, I just fundamentally don't think I'd want to be tested by an AI. I understand the author's reasoning, but it just doesn't feel respectful for something that is so high-stakes for the student. Wouldn't a written exam--or even a digital one, taken in class on school-provided machines--be almost as good? As long as it's not a hundred person class or something, you can also have an oral component t…

A written exam is problematic if you want the students to demonstrate mastery of the the content of their own project. It's also problematic if the course is essentially about using tools well. Bringing those tools into the exam without letting in LLMs is very hard.

I don't entirely disagree but all exams are problematic. We don't have the technology to look into a person's mind and see what they know. An exam is an imperfect data point.

Ask the student to come to the exam and write something new, which is similar to what they've been working on at home but not the same. You can even let them bring what they've done at home for reference, which will help if they actually understand what they've produced to date.

Re: Fighting Fire with Fire: Scalable Oral Exams

#32

Post by margalabargala »

Arodex wrote: post #30
dvh wrote: post #27

Students cheat when grades are more valuable than knowledge.

So, what is your solution to turn teenagers and 20-somethings into wise men and women?

Identifying a problem is the first step towards solving it. Coming up with a solution is a later step.

Re: Fighting Fire with Fire: Scalable Oral Exams

#33

Post by skywalqer »

eaglefield wrote: post #29

At the price per student it probably makes sense to run some voluntary trial exams during the semester. This would give students a chance to get acquainted to the format, help them check their understanding and if the voice is very intimidating allow them to get used to that as well. As an aside, I'm surprised oral exams aren't possible at 36 students. I feel like I've taken plenty of courses with more participants a…

At my university (Charles University in Prague), we had oral exams for 200+ people (spread over many different sessions).

Re: Fighting Fire with Fire: Scalable Oral Exams

#34

Post by throwaway7783 »

Wowfunhappy wrote: post #8

...if I was a student, I just fundamentally don't think I'd want to be tested by an AI. I understand the author's reasoning, but it just doesn't feel respectful for something that is so high-stakes for the student. Wouldn't a written exam--or even a digital one, taken in class on school-provided machines--be almost as good? As long as it's not a hundred person class or something, you can also have an oral component t…

Why is it disrespectful? It is just a task. And it is almost an arms race b/w students and profs. Has always been (smuggling written notes into the exam etc)

Re: Fighting Fire with Fire: Scalable Oral Exams

#36

Post by Arodex »

eaglefield wrote: post #29

At the price per student it probably makes sense to run some voluntary trial exams during the semester. This would give students a chance to get acquainted to the format, help them check their understanding and if the voice is very intimidating allow them to get used to that as well. As an aside, I'm surprised oral exams aren't possible at 36 students. I feel like I've taken plenty of courses with more participants a…

>As an aside, I'm surprised oral exams aren't possible at 36 students.

It depends on how frequent and how in-depth you want the exams to be. How much knowledge can you test in an oral exam that would be similar to a two-hour written exam? (Especially when I remember my own experience where I would have to sketch ideas for 3/4th of the time alloted before spending the last 1/4th writing frenetically the answer I found _in extremis_).

If I were a teacher, my experience would be to sample the students. Maybe bias the sample towards students who give wrong answers, but then it could start either a good feedback loop ("I'll study because I don't want to be interrogated again in front of the class") or a bad feedback loop ("I am being picked on, it is getting worse than I can improve, I hate this and I give up")

Re: Fighting Fire with Fire: Scalable Oral Exams

#37

Post by eaglefield »

skywalqer wrote: post #33
eaglefield wrote: post #29

At the price per student it probably makes sense to run some voluntary trial exams during the semester. This would give students a chance to get acquainted to the format, help them check their understanding and if the voice is very intimidating allow them to get used to that as well. As an aside, I'm surprised oral exams aren't possible at 36 students. I feel like I've taken plenty of courses with more participants a…

At my university (Charles University in Prague), we had oral exams for 200+ people (spread over many different sessions).

Impressive!

I think the most I experienced at the physics department in Aarhus was 70ish students. 200 sounds like a big undertaking.

Re: Fighting Fire with Fire: Scalable Oral Exams

#38

Post by throwaway7783 »

anonymous908213 wrote: post #26
kelseyfrog wrote: post #15

Earlier quoted context omitted.

If I was a professor, I don't think I'd want students submitting AI generated work. Yet, here we are. Students had and still have the option to collectively choose not to use AI to cheat. We can go back to written work at any time. And yet they continue to use it. Curious.

Ah yes, collective punishment. Exactly what we should be endeavouring for our professors to do: see the student as an enemy to be disciplined, not a mind to be nurtured. I know we've had historical record of people saying this for 2000 years and counting, but I suspect the future is well and truly bleak. Not because of the next generation of students, but because of the current generation of educators unable to succe…

Since when did exams become punishment? Aren't they a reflection of what you have learnt as imperfect as they are?

Re: Fighting Fire with Fire: Scalable Oral Exams

#39

Post by senko »

margalabargala wrote: post #32
Arodex wrote: post #30

Earlier quoted context omitted.

So, what is your solution to turn teenagers and 20-somethings into wise men and women?

Identifying a problem is the first step towards solving it. Coming up with a solution is a later step.

Very insightful!

Here, I'll identify another: There is much pain and suffering in this world.

Coming up with a solution is left as an excercise for the reader.

Re: Fighting Fire with Fire: Scalable Oral Exams

#40

Post by Wowfunhappy »

throwaway7783 wrote: post #34
Wowfunhappy wrote: post #8

...if I was a student, I just fundamentally don't think I'd want to be tested by an AI. I understand the author's reasoning, but it just doesn't feel respectful for something that is so high-stakes for the student. Wouldn't a written exam--or even a digital one, taken in class on school-provided machines--be almost as good? As long as it's not a hundred person class or something, you can also have an oral component t…

Why is it disrespectful? It is just a task. And it is almost an arms race b/w students and profs. Has always been (smuggling written notes into the exam etc)

The student has a lot riding on the outcome of their exam. The teacher is making a black box of nondeterministic matrix multiplication at least partially responsible for that outcome. Sure, the AI isn't the one grading, but it is deciding which questions and follow up questions to ask.

Let me ask, how do you generally feel when you contact customer service about something and you get an AI chatbot? Now imagine the chatbot is responsible for whether you pass the course.

Post reply on HN