A regular paper and pencil exam would be a better experience for the students.
Fighting Fire with Fire: Scalable Oral Exams
141–150 of 288 posts
Re: Fighting Fire with Fire: Scalable Oral Exams
#142Earlier quoted context omitted.
One student had to talk to an AI for more than 60 minutes. These guys are creating a dystopia. Also students will just have an AI pick up the phone if this gets used for more than 2 semesters.
Regular exams definitely take more than a single hour though. How is this bad?
Talking to inanimate objects is for 5-year-olds and the mentally ill.
Re: Fighting Fire with Fire: Scalable Oral Exams
#143Earlier quoted context omitted.
One way of scaling out interactive/oral assessment (and personalized instruction in general) is to hire a group of course assistants/tutors from the previous cohort.
So, TAs. The other half of the mission-critical staff that keeps a university running.
I think it works differently at different schools and in different countries, but hourly (often undergraduate work-study) course assistants in the US can be very affordable since they typically still pay tuition and are paid at a lower rate than fully funded (usually graduate student) TAs.
Re: Fighting Fire with Fire: Scalable Oral Exams
#144> We surveyed students before releasing grades to capture their experience. [...] Only 13% preferred the AI oral format. 57% wanted traditional written exams. [...] 83% of students found the oral exam framework more stressful than a written exam. [...] > Take-home exams are dead. Reverting to pen-and-paper exams in the classroom feels like a regression. Yeah, not sure the conclusion of the article really matches the…
The issue is that it is not scalable, unless there is some dependable, automated way to convert handwriting to text.
[deleted]
Re: Fighting Fire with Fire: Scalable Oral Exams
#145Earlier quoted context omitted.
One student had to talk to an AI for more than 60 minutes. These guys are creating a dystopia. Also students will just have an AI pick up the phone if this gets used for more than 2 semesters.
It's not that the oral format should be dismissed, just that the idea of your exam being speaking to a machine to be judged on the merit of your time in a course is dystopian. Talking to another human is fine.
How different is it in essence from checking boxes to be scanned by a machine and auto-evaluated to get a one dimention numerical score ?
Have exams ever been about humanity and the optics of it ?
Re: Fighting Fire with Fire: Scalable Oral Exams
#146Earlier quoted context omitted.
University exams being marked by hand, by someone experienced enough to work outside a rigid marking scheme, has been the standard for hundreds of years and has proven scalable enough. If there are so many students that academics can’t keep up, there are likely too many students to maintain a high standard of education anyway.
A limitation of written exams is in distance education, which simply was hardly a thing for the hundreds of years exams were used. Just like WFH is a new practice employers have to learn to deal with, study from home (SFH) is a phenomenon that is going to affect education. The objections to SFH exist and are strikingly similar to objections to WFH, but the economics are different. Some universities already see value…
Distance education is a tiny percentage of higher education though. Online classes at a local university are more common, but you can still bring the students in for proctored exams.
Even for distance education though, proctored testing centers have been around longer than the internet.
Re: Fighting Fire with Fire: Scalable Oral Exams
#147This is all so crazy to me. I went to school long before LLMs were even a Google Engineer's brianfart for the transformer paper and the way I took exams was already AI proof. Everything hand written in pen in a proctored gymnasium. No open books. No computers or smart phones, especially ones connected to the internet. Just a department sanctioned calculator for math classes. I wrote assembly and C++ code by hand, and…
I simply don't believe your university program had a 99% failure rate. Such a university should be shut down and sold for parts.
Re: Fighting Fire with Fire: Scalable Oral Exams
#148This is all so crazy to me. I went to school long before LLMs were even a Google Engineer's brianfart for the transformer paper and the way I took exams was already AI proof. Everything hand written in pen in a proctored gymnasium. No open books. No computers or smart phones, especially ones connected to the internet. Just a department sanctioned calculator for math classes. I wrote assembly and C++ code by hand, and…
What's the crazy to me is you took that as the gold standard for education evaluation.
For comparison we had lengthy sessions in a jailed terminal, week after week, writing C programs covering specific algorithms, compiling and debugging them within these sessions and assistants would follow our progress and check we're getting it. Those not finishing in time get additional sessions.
Last exam was extremely simple and had very little weight in the overall evaluation.
That might not scale as much, but that's definitely what I'd long for, not the Chuck Norris style cram school exam you are drawing us.
Re: Fighting Fire with Fire: Scalable Oral Exams
#149This is all so crazy to me. I went to school long before LLMs were even a Google Engineer's brianfart for the transformer paper and the way I took exams was already AI proof. Everything hand written in pen in a proctored gymnasium. No open books. No computers or smart phones, especially ones connected to the internet. Just a department sanctioned calculator for math classes. I wrote assembly and C++ code by hand, and…
Admitting 1000 students to get 10 graduates means there are morons in admissions doing zero vetting to make sure the students are qualified.
Absolutely not morons. If the goal is to maximize collecting tuition and still have reputation of not being a diploma shop this is the obvious solution. The 20% which survives the first year is worth keeping around to hire them later in the companies which the teaching staff own or collect referral bonuses if working for a multinational.
Re: Fighting Fire with Fire: Scalable Oral Exams
#150Earlier quoted context omitted.
What's stopping you from just using the AI to directly accomplish the ultimate goal, rather than taking the very indirect route of educating humans to do it?
What's the end vision here? A society of useless, catatonic humans taken care of by a superintelligence? Even if that's possible, I wouldn't call that desirable. Education is fundamental for raising competent adults.
Great question about what adults can be more competent about than an artificial superintelligence. ‘How to be a human’ comes to mind and not much more.