Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fighting Fire with Fire: Scalable Oral Exams

behind-the-enemy-lines.com

Post reply on HN
Print view View original

111–120 of 288 posts

Re: Fighting Fire with Fire: Scalable Oral Exams

#111

Post by semilin »

dvh wrote: post #27

Students cheat when grades are more valuable than knowledge.

I think this points to the only real sustainable solution: make it so that students would prefer to do real work. We have seen for ages the distinction between seeming and being in regards to verbal understanding blurred. LLMs are only an acceleration of the blurring. Therefore it will at some point become essentially impossible to determine whether one really understands something.

The two solutions to this are (1) as some commenters here are suggesting, give up entirely and focus only on quality of output, or (2) teach students to care about being more than appearance. Make students want to write essays. It is for their personal edification and intellectual flourishing. The benefits of this far surpass output.

Obviously this is an enormously difficult task, but let us not suppose it an unworthy one.

Re: Fighting Fire with Fire: Scalable Oral Exams

#112

Post by phren0logy »

I had plenty of oral exams throughout my education and training. It's interesting to see their resurgence, and easy to understand the appeal. If they can be done rigorously and fairly (no easy thing), then they go much further than multiple can in demonstrating understanding of concepts. But, they are inherently more stressful. I agree with the article that the increased pressure is a feature, not a bug. It's much more real-world for many kinds of knowledge.

Re: Fighting Fire with Fire: Scalable Oral Exams

#113

Post by jimbokun »

throwaway7783 wrote: post #34
Wowfunhappy wrote: post #8

...if I was a student, I just fundamentally don't think I'd want to be tested by an AI. I understand the author's reasoning, but it just doesn't feel respectful for something that is so high-stakes for the student. Wouldn't a written exam--or even a digital one, taken in class on school-provided machines--be almost as good? As long as it's not a hundred person class or something, you can also have an oral component t…

Why is it disrespectful? It is just a task. And it is almost an arms race b/w students and profs. Has always been (smuggling written notes into the exam etc)

Talking to a disembodied inhuman voice can be disconcerting and produce anxiety in a way that wouldn’t be true communicating to a live human instructor.

Adding this as an additional optional tool, though, is an excellent idea.

Re: Fighting Fire with Fire: Scalable Oral Exams

#114

Post by jimbokun »

viccis wrote: post #48
throwaway7783 wrote: post #34

Earlier quoted context omitted.

Why is it disrespectful? It is just a task. And it is almost an arms race b/w students and profs. Has always been (smuggling written notes into the exam etc)

Unless class sizes are astronomical, it's absurd to pay US tuition all to have a lazy professor who automates even the most human components of the education you're getting for that price. If the class cost me $50? Then sure, use Dr. Slop to examine my knowledge. But this professor's school charges them $90,000 a year and over $200k to get an MBA? Hell no!

Yes.

At that point what’s the value add over using YouTube videos and ChatGPT on your own?

Re: Fighting Fire with Fire: Scalable Oral Exams

#116

Post by jimbokun »

kelseyfrog wrote: post #15
Wowfunhappy wrote: post #8

...if I was a student, I just fundamentally don't think I'd want to be tested by an AI. I understand the author's reasoning, but it just doesn't feel respectful for something that is so high-stakes for the student. Wouldn't a written exam--or even a digital one, taken in class on school-provided machines--be almost as good? As long as it's not a hundred person class or something, you can also have an oral component t…

If I was a professor, I don't think I'd want students submitting AI generated work. Yet, here we are. Students had and still have the option to collectively choose not to use AI to cheat. We can go back to written work at any time. And yet they continue to use it. Curious.

And instructors also have the option to not have AI do their work.

Re: Fighting Fire with Fire: Scalable Oral Exams

#117

Post by semilin »

jimbokun wrote: post #110
Wowfunhappy wrote: post #8

...if I was a student, I just fundamentally don't think I'd want to be tested by an AI. I understand the author's reasoning, but it just doesn't feel respectful for something that is so high-stakes for the student. Wouldn't a written exam--or even a digital one, taken in class on school-provided machines--be almost as good? As long as it's not a hundred person class or something, you can also have an oral component t…

I would be annoyed that I can’t use AI to do my work but the instructor can have AI do his job.

Too bad. The premise should be that the instructor, by nature of having the position, already has understanding of the subject. As a student, you do not, and your goal is to gain it. Prompting an LLM to write a response for you does not build understanding. Therefore you should write unhindered by sophistry machines.

Re: Fighting Fire with Fire: Scalable Oral Exams

#118

Post by vkou »

NewsaHackO wrote: post #71
michaelt wrote: post #62

> We surveyed students before releasing grades to capture their experience. [...] Only 13% preferred the AI oral format. 57% wanted traditional written exams. [...] 83% of students found the oral exam framework more stressful than a written exam. [...] > Take-home exams are dead. Reverting to pen-and-paper exams in the classroom feels like a regression. Yeah, not sure the conclusion of the article really matches the…

The issue is that it is not scalable, unless there is some dependable, automated way to convert handwriting to text.

I assure you, oral exams are completely scalable. But it does require most of a university's budget to go towards labs and faculty, and not administration and sports arenas and social services and vanity projects and three-star dorms.

Re: Fighting Fire with Fire: Scalable Oral Exams

#119

Post by neilv »

Instead of funneling more business/hype to the AI bro industry, to police the AI bro industry that fully expected this effect from their cheating-on-your-homework/plagiarism services (oh, I see this is a business school)...

First, the business school administration and faculty firmly commits, that plagiarism, including with AI, means prompt dismissal.

Then, the first time you have a suspicion of plagiarism, you investigate.

After the first student of a class year is found guilty, and smacked to curb, all the other students will know, and I bet your problem is mostly solved for that class year.

Then, one coked-up nepo baby sociopath will think they are too smart or meritorious to "fail" by getting caught. Bam! Smacked to the curb.

Then one of those two will try sue, and the university PR professionals will laugh at them, for putting their name in the news as someone who got kicked out of business school for cheating. The business school will take this opportunity to bolster their reputation for excellence.

At this point, it will become standard advice for the subsequent class years, that cheating at this school is something only an idiot loser does, not a winner MBA.

Re: Fighting Fire with Fire: Scalable Oral Exams

#120

Post by vasco »

michaelt wrote: post #62

> We surveyed students before releasing grades to capture their experience. [...] Only 13% preferred the AI oral format. 57% wanted traditional written exams. [...] 83% of students found the oral exam framework more stressful than a written exam. [...] > Take-home exams are dead. Reverting to pen-and-paper exams in the classroom feels like a regression. Yeah, not sure the conclusion of the article really matches the…

One student had to talk to an AI for more than 60 minutes. These guys are creating a dystopia. Also students will just have an AI pick up the phone if this gets used for more than 2 semesters.
Post reply on HN