Fighting Fire with Fire: Scalable Oral Exams • Hacker News Board

Re: Fighting Fire with Fire: Scalable Oral Exams

#71

Post by NewsaHackO » Fri, Jan 02, 2026, 9:14 PM UTC

michaelt wrote: post #62

> We surveyed students before releasing grades to capture their experience. [...] Only 13% preferred the AI oral format. 57% wanted traditional written exams. [...] 83% of students found the oral exam framework more stressful than a written exam. [...] > Take-home exams are dead. Reverting to pen-and-paper exams in the classroom feels like a regression. Yeah, not sure the conclusion of the article really matches the…

The issue is that it is not scalable, unless there is some dependable, automated way to convert handwriting to text.

Re: Fighting Fire with Fire: Scalable Oral Exams

#72

Post by Wowfunhappy » Fri, Jan 02, 2026, 9:14 PM UTC

lifetimerubyist wrote: post #60

This is all so crazy to me. I went to school long before LLMs were even a Google Engineer's brianfart for the transformer paper and the way I took exams was already AI proof. Everything hand written in pen in a proctored gymnasium. No open books. No computers or smart phones, especially ones connected to the internet. Just a department sanctioned calculator for math classes. I wrote assembly and C++ code by hand, and…

I basically agree with the thrust of what you're saying, but also:

> I wrote assembly and C++ code by hand, and it was expected to compile. No, I never got a chance to try to compile it myself before submitting it for grading.

Do you, like, really think this is the best way to assess someone's ability? Can't we find a place between the two extremes?

Personally, I'd go with a school-provided computer with a development environment and access to documentation. No LLMs, except maybe (but probably not) for very high-level courses.

Re: Fighting Fire with Fire: Scalable Oral Exams

#73

Post by Aurornis » Fri, Jan 02, 2026, 9:16 PM UTC

dvh wrote: post #27

Students cheat when grades are more valuable than knowledge.

I knew some hardcore, dedicated cheaters in college. All of them hit a wall where their cheating tricks stopped working. Most of them couldn't get back on track.

I suppose there are other fields where the degree might be used mostly as a filtering mechanism, where cheating through graduation might get you a job doing work different than your classes anyway. However, even in those cases it's hard to break the habit of cheating your way around every difficult problem that comes your way.

Re: Fighting Fire with Fire: Scalable Oral Exams

#74

Post by Kwpolska » Fri, Jan 02, 2026, 9:18 PM UTC

NewsaHackO wrote: post #71

michaelt wrote: post #62

> We surveyed students before releasing grades to capture their experience. [...] Only 13% preferred the AI oral format. 57% wanted traditional written exams. [...] 83% of students found the oral exam framework more stressful than a written exam. [...] > Take-home exams are dead. Reverting to pen-and-paper exams in the classroom feels like a regression. Yeah, not sure the conclusion of the article really matches the…

The issue is that it is not scalable, unless there is some dependable, automated way to convert handwriting to text.

Why is this a problem now, but was not a problem for the past few centuries? This class had 36 students, you could grade that in a single evening.

Re: Fighting Fire with Fire: Scalable Oral Exams

#75

Post by schainks » Fri, Jan 02, 2026, 9:22 PM UTC

My Italian friends went through only oral exams in high school and it worked very well for them.

The key implementation detail to me is that the whole class is sitting in on your exam (not super scalable, sure) so you are literally proving to your friends you aren’t full of shit when doing an exam.

Re: Fighting Fire with Fire: Scalable Oral Exams

#76

Post by pgalvin » Fri, Jan 02, 2026, 9:22 PM UTC

NewsaHackO wrote: post #71

michaelt wrote: post #62

> We surveyed students before releasing grades to capture their experience. [...] Only 13% preferred the AI oral format. 57% wanted traditional written exams. [...] 83% of students found the oral exam framework more stressful than a written exam. [...] > Take-home exams are dead. Reverting to pen-and-paper exams in the classroom feels like a regression. Yeah, not sure the conclusion of the article really matches the…

The issue is that it is not scalable, unless there is some dependable, automated way to convert handwriting to text.

University exams being marked by hand, by someone experienced enough to work outside a rigid marking scheme, has been the standard for hundreds of years and has proven scalable enough. If there are so many students that academics can’t keep up, there are likely too many students to maintain a high standard of education anyway.

Re: Fighting Fire with Fire: Scalable Oral Exams

#77

Post by perching_aix » Fri, Jan 02, 2026, 9:22 PM UTC

lifetimerubyist wrote: post #60

This is all so crazy to me. I went to school long before LLMs were even a Google Engineer's brianfart for the transformer paper and the way I took exams was already AI proof. Everything hand written in pen in a proctored gymnasium. No open books. No computers or smart phones, especially ones connected to the internet. Just a department sanctioned calculator for math classes. I wrote assembly and C++ code by hand, and…

> Cohorts for programs with a thousand initial students had less than 10 graduates. This was the norm.

And why is this a flex exactly? Almost sounds like fraud. Get sold on how you'll be taught well and become successful. Pay. Then be sent through an experience that filters so severely, only 1% of people pass. Receive 100% of the blame when you inevitably fail. Repeat for the other 990 students. The "university thanks you for your donation" slogan doesn't sound too hot all of a sudden.

It's like some malicious compliance take on both teaching and studying. Which shouldn't even be surprising, considering the circumstances of the professors e.g. where I studied, as well as the students'.

Mind you, I was (for some classes) tested the same way. People still cheated, and grading stringency varied. People still also forgot everything shortly after wrapping up their finals on the given subjects and moved on. People also memorized questions and compiled a solutions book, and then handed them down to next year's class. Because this method does jack against that on its own. You still need to keep crafting novel questions, vary them more than just by swapping key values, etc.

Re: Fighting Fire with Fire: Scalable Oral Exams

#78

Post by Aurornis » Fri, Jan 02, 2026, 9:23 PM UTC

NewsaHackO wrote: post #71

michaelt wrote: post #62

> We surveyed students before releasing grades to capture their experience. [...] Only 13% preferred the AI oral format. 57% wanted traditional written exams. [...] 83% of students found the oral exam framework more stressful than a written exam. [...] > Take-home exams are dead. Reverting to pen-and-paper exams in the classroom feels like a regression. Yeah, not sure the conclusion of the article really matches the…

The issue is that it is not scalable, unless there is some dependable, automated way to convert handwriting to text.

When college degrees cost as much as they do, it's reasonable to pay people to do the transcription and/or grading.

Work study and TA jobs were abundant when I was in college. It wasn't a problem in the past and shouldn't be a problem now.

Re: Fighting Fire with Fire: Scalable Oral Exams

#79

Post by NewsaHackO » Fri, Jan 02, 2026, 9:23 PM UTC

acbart wrote: post #61

lifetimerubyist wrote: post #60

This is all so crazy to me. I went to school long before LLMs were even a Google Engineer's brianfart for the transformer paper and the way I took exams was already AI proof. Everything hand written in pen in a proctored gymnasium. No open books. No computers or smart phones, especially ones connected to the internet. Just a department sanctioned calculator for math classes. I wrote assembly and C++ code by hand, and…

I've had colleagues argue (prior to LLMs) that oral exams are superior to paper exams, for diagnosing understanding. I don't know how to validate that statement, but if the assumption is true than there is merit to finding a way to scale them. Not saying this is it, but I wouldn't say that it's fair to just dismiss oral exams entirely.

Yes, I hate oral exams, but they are definitely better at getting a whole picture of a person's understanding of topics. A lot of specialty boards in medicine do this. To me, the two issues are that it requires an experienced, knowledgeable, and empathetic examiner, who is able to probe the examinee about areas they seem to be struggling in, and paradoxically, its strength is in the fact that it is subjective. The examiner may have set questions, but how the examinee answers the questions and the follow-up questions are what differentiate it from a written exam. If the examiner is just the equivalent of a customer service representative and is strictly following a tree of questions, it loses its value.

Re: Fighting Fire with Fire: Scalable Oral Exams

#80

Post by [deleted] » Fri, Jan 02, 2026, 9:23 PM UTC

kelseyfrog wrote: post #56

Wowfunhappy wrote: post #23

Earlier quoted context omitted.

> Students had and still have the option to collectively choose not to use AI to cheat. Individuals can't "collectively" choose anything. This test is given to the entire class, including people who never touched AI.

What are you talking about? Students could absolutely organize a consensus decision to not use AI. People do this all the time. How do you think human organizations continue to exist?

[deleted]