Live data from Hacker News

Fighting Fire with Fire: Scalable Oral Exams

behind-the-enemy-lines.com

101–110 of 288 posts

Re: Fighting Fire with Fire: Scalable Oral Exams

#101
post #52

Just let students use whatever tool they want and make them compete for top grades. Distribution curving is already normal in education. If an AI answer is the grading floor, whatever they add will be visible signal. People who just copy and paste a lame prompt will rank at the bottom and fail without any cheating gymnastics. Plus this is more like how people work. https://sibylline.dev/articles/2025-12-31-how-agent-…

> Plus this is more like how people work. if we want to educate people 'how people work', companies should be hiring interns and teaching them how people work. university education should be about education (duh) and deep diving into a few specialized topics, not job preparedness. AI makes this disconnect that much more obvious.

If that was the model all but a small handful of universities would be shut down tomorrow. It’s impossible to fund that many university degrees without the promise of increased earnings after completion.

Re: Fighting Fire with Fire: Scalable Oral Exams

#102
post #2

If you can use AI agents to give exams, what is stopping you from using them to teach the whole course? Also, with all the progress in video gen, what does recording the webcam really do?

What's stopping you from just using the AI to directly accomplish the ultimate goal, rather than taking the very indirect route of educating humans to do it?

What's the end vision here? A society of useless, catatonic humans taken care of by a superintelligence? Even if that's possible, I wouldn't call that desirable. Education is fundamental for raising competent adults.

Re: Fighting Fire with Fire: Scalable Oral Exams

#103
post #7

> We can publish exactly how the exam works—the structure, the skills being tested, the types of questions. No surprises. The LLM will pick the specific questions live, and the student will have to handle them. I wonder: with a structure like this, it seems feasible to make the LLM exam itself available ahead of time, in its full authentic form. They say the topic randomization is happening in code, and that this who…

It does sound like an excellent teaching tool.

To the extent of wondering what value the human instructors add.

Re: Fighting Fire with Fire: Scalable Oral Exams

#105

This is all so crazy to me. I went to school long before LLMs were even a Google Engineer's brianfart for the transformer paper and the way I took exams was already AI proof. Everything hand written in pen in a proctored gymnasium. No open books. No computers or smart phones, especially ones connected to the internet. Just a department sanctioned calculator for math classes. I wrote assembly and C++ code by hand, and…

[deleted]

Re: Fighting Fire with Fire: Scalable Oral Exams

#107

This is all so crazy to me. I went to school long before LLMs were even a Google Engineer's brianfart for the transformer paper and the way I took exams was already AI proof. Everything hand written in pen in a proctored gymnasium. No open books. No computers or smart phones, especially ones connected to the internet. Just a department sanctioned calculator for math classes. I wrote assembly and C++ code by hand, and…

I basically agree with the thrust of what you're saying, but also: > I wrote assembly and C++ code by hand, and it was expected to compile. No, I never got a chance to try to compile it myself before submitting it for grading. Do you, like, really think this is the best way to assess someone's ability? Can't we find a place between the two extremes? Personally, I'd go with a school-provided computer with a developmen…

The safe middle space still does not involve a computer

Lots of my tests involved writing pseudocode, or "Just write something that looks like C or Java". Don't miss the semicolon at the end of the line, but if you write "System.print()" rather than "System.out.printLn()" you might lose a single point. Maybe.

If there were specific functions you need to call, it would have a man page or similar on the test itself, or it would be the actual topic under test.

I hand wrote a bunch of SQL queries. Hand wrote code for my Systems Programming class that involved pointers. I'm not even good with pointers. I hand wrote Java for job interviews.

It's pretty rare that you need to actually test someone can memorize syntax, that's like the entire point of modern development environments.

But if you are completely unable to function without one, you might not know as much as you would hope.

The first algorithms came before the first programming languages.

Sure, it means you need to be able to run the code in your head and be able to mentally "debug" it, but that's a feature

If you could not manage these things, you washed out in the CS101 class that nearly every STEM student took. The remaining students were not brilliant, but most of them could write code to solve problems. Then you got classes that could actually teach and test that problem solving itself.

The one class where we built larger apps more akin to actual jobs, that could have been done entirely in the lab with locked down computers if need be, but the professor really didn't care if you wanted to fake the lab work, you still needed to pass the book learning for "Programming Patterns" which people really struggled with and you still needed to be able to give a "Demo" and presentation, and you still needed to demonstrate that you understood how to read some requests from a "Customer" and turn it into features and requirements and UX

Nobody cares about people sabotaging their own education except in programming because no matter how much MBAs insist that all workers are replaceable, they cannot figure out a way to actually evaluate the competency of a programmer without knowing programming. If an engineer doesn't actually understand how to evaluate static stresses on a structure, they are going to have a hard time keeping a job. Meanwhile in the world of programming, hopping around once a year is "normal" somehow, so you can make a lot of money while literally not knowing fizzbuzz. I don't think the problem is actually education.

Computer Science isn't actually about using a laptop.

Re: Fighting Fire with Fire: Scalable Oral Exams

#108
post #27

Students cheat when grades are more valuable than knowledge.

This is not hitting the problem. Most students in universities are completely fine with awful grades or expect comical levels of grade inflation. Ask a professor or TA and you'll hear about an insane level of entitlement from students after they hand in extremely shoddy work. Failing students is actually quite hard or extremely discouraged by admins. The real problem is students and universities have collectively bou…

Universities are in for a rude awakening when employers realize their degrees mean nothing, stop hiring their graduates, and then students stop enrolling.

Re: Fighting Fire with Fire: Scalable Oral Exams

#109
post #3

It's dehumanizing to be grilled by AI, whether it is a job interview or a university exam. ...but OTOH if cheating is so easy it's impossible to resist and when everyone cheats honest students are the ones getting all the bad grades, what else can you do?

Written exams at a set time and place graded by a human grader.

Re: Fighting Fire with Fire: Scalable Oral Exams

#110

...if I was a student, I just fundamentally don't think I'd want to be tested by an AI. I understand the author's reasoning, but it just doesn't feel respectful for something that is so high-stakes for the student. Wouldn't a written exam--or even a digital one, taken in class on school-provided machines--be almost as good? As long as it's not a hundred person class or something, you can also have an oral component t…

I would be annoyed that I can’t use AI to do my work but the instructor can have AI do his job.
Post reply on HN