Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fighting Fire with Fire: Scalable Oral Exams

behind-the-enemy-lines.com

Post reply on HN
Print view View original

131–140 of 288 posts

Re: Fighting Fire with Fire: Scalable Oral Exams

#131

Post by philipallstar »

> I had prepared thoroughly and felt confident in my understanding of the material, but the intensity of the interviewer's voice during the exam unexpectedly heightened my anxiety and affected my performance. The experience was more triggering than I anticipated, which made it difficult to fully demonstrate my knowledge. Throughout the course, I have actively participated and engaged with the material, and I had hoped to better demonstrate my knowledge in this interview.

This sounds as though it was written by an LLM too.

Re: Fighting Fire with Fire: Scalable Oral Exams

#132

Post by aaplok »

pgalvin wrote: post #76
NewsaHackO wrote: post #71

Earlier quoted context omitted.

The issue is that it is not scalable, unless there is some dependable, automated way to convert handwriting to text.

University exams being marked by hand, by someone experienced enough to work outside a rigid marking scheme, has been the standard for hundreds of years and has proven scalable enough. If there are so many students that academics can’t keep up, there are likely too many students to maintain a high standard of education anyway.

A limitation of written exams is in distance education, which simply was hardly a thing for the hundreds of years exams were used. Just like WFH is a new practice employers have to learn to deal with, study from home (SFH) is a phenomenon that is going to affect education.

The objections to SFH exist and are strikingly similar to objections to WFH, but the economics are different. Some universities already see value in offering that option, and they (of course) leave it to the faculty to deal with the consequences.

Re: Fighting Fire with Fire: Scalable Oral Exams

#133

Post by vkou »

musicale wrote: post #122
vkou wrote: post #118

Earlier quoted context omitted.

I assure you, oral exams are completely scalable. But it does require most of a university's budget to go towards labs and faculty, and not administration and sports arenas and social services and vanity projects and three-star dorms.

One way of scaling out interactive/oral assessment (and personalized instruction in general) is to hire a group of course assistants/tutors from the previous cohort.

So, TAs. The other half of the mission-critical staff that keeps a university running.

Re: Fighting Fire with Fire: Scalable Oral Exams

#134

Post by knallfrosch »

YakBizzarro wrote: post #11

I seriously don't get it. At my time in university, ALL the exams were oral. And most had one or two written parts before (one even three, the professor called it written-for-the-oral). Sure, the orals took two days for the big exams at the beginning, still, professors and their assistants managed to offer six sessions per year.

Professors are just humans. If they can grade you with an AI for $5 and spend the 20 hours gained scrolling on their phone – guess what, they'll do that.

Re: Fighting Fire with Fire: Scalable Oral Exams

#135

Post by perching_aix »

jmye wrote: post #89
perching_aix wrote: post #77

Earlier quoted context omitted.

> Cohorts for programs with a thousand initial students had less than 10 graduates. This was the norm. And why is this a flex exactly? Almost sounds like fraud. Get sold on how you'll be taught well and become successful. Pay. Then be sent through an experience that filters so severely, only 1% of people pass. Receive 100% of the blame when you inevitably fail. Repeat for the other 990 students. The "university thank…

> And why is this a flex exactly? Almost sounds like fraud. Do you think you're just purchasing a diploma? Or do you think you're purchasing the opportunity to gain an education and potential certification that you received said education? It's entirely possible that the University stunk at teaching 99% of it's students (about as equally possible that 99% of the students stunk at learning), but "fraud" is absolute no…

I don't think one applies to university expecting they're purchasing themselves a diploma, nor that they should be magically absolved of putting in effort to learn the material. What I do think is that the place they describe sounds an awful lot like people being set up for failure though, and so that begged the question as to why that might be. I should probably clarify that I wasn't particularly serious about my fraud suggestion however (was just a bit of a jab rather), as that doesn't seem to have made it through.

If teaching was so simple that you could just tell people to go RTFM then recite it from memory, I don't know why people are bothering with pedagogy at all. It'd seem that there's more to teaching and learning than the bare minimum, and that both parties are culpable. Doesn't sound like you disagree on that either.

> you're purchasing the opportunity to

We can swap out fraud for gambling if you like :) Sounds like an even closer analogy now that you mention!

Jokes aside though, isn't it a gamble? You gamble with yourself that you can [grow to] endure and succeed or drop out / something worse. The stake is the tuition, the prize is the diploma.

Now of course, tuition is per semester (here at least, dunno elsewhere), so it's reasonable to argue that the financial investment is not quite in such jeopardy as I painted it. Not sure about the emotional investment though.

Consider the Chinese Gaokao exam, especially in its infamous historical context between the 70s and 90s. The number of available seats was way lower than the number of applications [0]. The exams grueling. What do you reckon, was it the people's fault for not winning an essentially unspoken lottery? Who do you think received the blame? According to a cursory search, the individual and their families (wasn't there, cannot know) received the blame. And no, I don't think in such a tortured scheme it is the students' fault for not making the bar.

If there are fewer seats than what there is demand for, then that's overbooking, and you the test authoring / conducting authority are biased to artificially induce test failures. It is no longer a fair assessment, nor a fair dynamic. Conversely, passing is no longer an honest signal of qualification. Or rather, not passing is no longer an honest signal of unqualification. And this doesn't have to come from a single test, it can be implemented structurally too, so that you shed people along the way. Which is what I'm actually alluding to.

[0] ~4.8%, so ~95% of people failed it by design: https://en.wikipedia.org/wiki/Class_of_1977%E2%80%931978_%28...

Re: Fighting Fire with Fire: Scalable Oral Exams

#136

Post by WJW »

vasco wrote: post #120
michaelt wrote: post #62

> We surveyed students before releasing grades to capture their experience. [...] Only 13% preferred the AI oral format. 57% wanted traditional written exams. [...] 83% of students found the oral exam framework more stressful than a written exam. [...] > Take-home exams are dead. Reverting to pen-and-paper exams in the classroom feels like a regression. Yeah, not sure the conclusion of the article really matches the…

One student had to talk to an AI for more than 60 minutes. These guys are creating a dystopia. Also students will just have an AI pick up the phone if this gets used for more than 2 semesters.

Regular exams definitely take more than a single hour though. How is this bad?

Re: Fighting Fire with Fire: Scalable Oral Exams

#137

Post by pamcake »

jimbokun wrote: post #83
lifetimerubyist wrote: post #60

This is all so crazy to me. I went to school long before LLMs were even a Google Engineer's brianfart for the transformer paper and the way I took exams was already AI proof. Everything hand written in pen in a proctored gymnasium. No open books. No computers or smart phones, especially ones connected to the internet. Just a department sanctioned calculator for math classes. I wrote assembly and C++ code by hand, and…

Admitting 1000 students to get 10 graduates means there are morons in admissions doing zero vetting to make sure the students are qualified.

There's either a 0 missing there or something pretty weird at that uni. I think the rest of the comment is very valid if we ignore this point.

My experience is the same except I think ~50% or so graduated[0].

[0]: Disclaimer that my programme was pretty competitive to get into, which is an earlier filter. Statistics looked worse for programmes at similar level with less people applying.

Re: Fighting Fire with Fire: Scalable Oral Exams

#138

Post by semilin »

jimbokun wrote: post #123
semilin wrote: post #117

Earlier quoted context omitted.

Too bad. The premise should be that the instructor, by nature of having the position, already has understanding of the subject. As a student, you do not, and your goal is to gain it. Prompting an LLM to write a response for you does not build understanding. Therefore you should write unhindered by sophistry machines.

But the instructor is not applying their understanding in any way. By delegating the evaluation to AI, there is zero value add vs just asking ChatGPT to evaluate your knowledge and not paying $1000s or $10000s in tuition. And universities wonder why enrollment is dropping.

I'm not intending to say it's acceptable for professors to use AI entirely in their grading. They obviously ought to contribute. I realize I actually misread your original comment, thinking of "instructor can have AI do his job" as "instructor can have AI to help do his job." Sorry about that. Point being, I think the expectation for real human thought ought to hold for both teacher and student.
Post reply on HN