Oral exams scale fine. A TA makes $25 per hour, and an oral exam is going to take an hour at most. I absolutely would not accept a $25 tuition rebate in exchange for having my exam administered by an LLM.
But you'll accept the results of an exam for a (in the US) $1000+ course given by a TA that makes about the same as a delivery driver? And you'll trust their assessment of the results? There's so much wrong with this idea, I don't even know where to start.
Obviously the session should be recorded & transcribed. If you take issue with your mark, you can escalate it to the professor, same as you would for a written exam.
If you're looking for suggestions, I'd love for you to start with a problem that isn't trivially fixable.
Lots of emotional commenting here. This guy, Panos Ipeirotis, is seriously on to the way university testing and corporate seminar testing will be done in the immediate future, as well as going forward. Complain all you want, this is inevitable. This initial version will improve. In time, more complex and multi-mod voice agents will do the teaching too, entirely individualized as well.
You know AI is a great solution that will succeed on its own merits when people need to be told it's "inevitable".
That's what so surprising to me - they data clearly shows the experiment had terrible results. And the write up is nothing but the author stating: "glowing success!". And they didn't even bother to test the most important thing. Were the LLM evaluations even accurate! Have graders manually evaluate them and see if the LLMs were even close or were wildly off. This is clearly someone who had a conclusion to promote reg…
At the risk of perhaps stating the obvious, there appears to be a whiff of aggression from this article. The "fighting fire with fire" language, the "haha, we love old FakeFoster, going to have to see if we change that" response to complaints that the voice was intimidating ... if there wasn't a specific desire to punish the class for LLM use by subjecting them to a robotic NKVD interrogation then the authors should…
The belligerence about changing the voice is so weird. And it does sort of set a tone straight off. "We got feedback that the voice was frightening and intimidating. We're keeping it tho."
> We surveyed students before releasing grades to capture their experience. [...] Only 13% preferred the AI oral format. 57% wanted traditional written exams. [...] 83% of students found the oral exam framework more stressful than a written exam. [...] > Take-home exams are dead. Reverting to pen-and-paper exams in the classroom feels like a regression. Yeah, not sure the conclusion of the article really matches the…
Not the case for the class in the blog post, but we also have many online classes. Many professionals prefer these online classes because they can attend without having to commute, and can do it from a place of their own convenience. Such classes do not have the luxury of pen-and-paper exams, and asking people to go to testing centers is a huge overkill. Take home exams for such settings (or any other form of written…
I did a C# module online run by a Norwegian University. It was worth 6 points, 180 grants you a bachelor's degree in Norway (or did, I think there have been changes since). The course ran over ten weeks and there were weekly assignments. Of course it would have been easy to cheat on those but there would be no point because there was a five hour invigilated open book exam at the end of the course. Had to go to a testing centre about 35 km away to take the exam but that really wasn't a great inconvenience. If I had wanted to pursue a whole degree then I would have had 30 such exams, roughly one a month if you do the degree over the traditional three years. That doesn't seem like overkill to me, it's a lot less effort than attending lectures and tutorials for three years as I did for my Applied Physics degree.
I'm afraid you misunderstand what it means to be "exempt" under the IRB. It doesn't mean "you don't have to talk to the IRB", it means "there's a little less oversight but you still need to file all the paperwork". Here's one university's explanation[1]: > Exempt human subjects research is a specific sub-set of “research involving human subjects” that does not require ongoing IRB oversight. Research can qualify for a…
The CFR is pretty clear, and I have experience with this (being both an IRB reviewer, faculty member, and researcher). When it says "is exempt" it means "is exempt". Imagine otherwise: a teacher who wants change their final exam from a 50 item Scantron using A-D choices, to a 50 item Scantron using A-E choices, because they think having 5 choices per item is better than 4, would need to ask for IRB approval. That's n…
You would be correct, except that this is a published blog post. It may not be in an academic journal, but this person has still conducted human subjects research that led to a published artifact. It was just "playing around" until they started posting their students' (summarized, anonymized) data to the internet.
University exams being marked by hand, by someone experienced enough to work outside a rigid marking scheme, has been the standard for hundreds of years and has proven scalable enough. If there are so many students that academics can’t keep up, there are likely too many students to maintain a high standard of education anyway.
The rate of college attendance has increased dramatically in the last 250 years, and especially in the last 75. In 1789 there were 1,000 enrolled college students total, in a country of 2.8M. In 2025, it is 19M students in a country of 340M. https://educationalpolicy.org/wp-content/uploads/2025/11/251... In 1950, 5.5% of adults ages 25-34 had completed a 4 year college degree. In 2018, it was 39%. https://www.highere…
So now we have a lot more people who can teach and mark exams.
Not the comprehensive physics exams I assigned as a prof. A well set exam takes at least 20-30 min to grade. That's 8-12 hours of work, and in practice, took several sittings over several days. If you are going to set an exam that can be graded in 5-10 min, you are not getting a lot of signal out of it. I wanted to do oral exams, but they are much more exhausting for the prof. Nominally, each student is with you for…
> If you are going to set an exam that can be graded in 5-10 min, you are not getting a lot of signal out of it. That's entirely false; this is why we have multiple-choice tests.
When I studied for my degree there were no multiple choice tests. In the final every question required a narrative answer justifying the conclusion.
Soo instead of solving the problem that the university supposedly doesn't have the money to have normal oral exams, they enshittified and techbrosified the entire process?
Thank god I had a chance to study in pre-AI times.
I assure you, oral exams are completely scalable. But it does require most of a university's budget to go towards labs and faculty, and not administration and sports arenas and social services and vanity projects and three-star dorms.
One way of scaling out interactive/oral assessment (and personalized instruction in general) is to hire a group of course assistants/tutors from the previous cohort.
As a student I really would not want to be taught by someone who was simply a couple of years ahead of me. I want my tutor to be a lot more experienced in both the subject and in tutoring.
One way of scaling out interactive/oral assessment (and personalized instruction in general) is to hire a group of course assistants/tutors from the previous cohort.
So, TAs. The other half of the mission-critical staff that keeps a university running.
Does any country other than the US use TAs? They certainly weren't a thing when I studied in the UK in the 1970s.