271–280 of 288 posts
Earlier quoted context omitted.
> spread over many different sessions this is also known as 'logistical nightmare', but yeah it's the only reasonable way if you want to avoid being questioned by robots.
Ah yes, the logistical nightmare any hair salon or nail studio handles just fine.
This is to be expected. The big commercial LLMs generally respond with text that agrees with the user.
> But here's what's interesting: the disagreement wasn't random. Problem Framing and Metrics had 100% agreement within 1 point. Experimentation? Only 57%.
> Why? When students give clear, specific answers, graders agree. When students give vague hand-wavy answers, graders (human or AI) disagree on how much partial credit to give. The low agreement on experimentation reflects genuine ambiguity in student responses, not grader noise.
The disagreement between the LLMs is interesting. I would hesitate to conclude that "low agreement on experimentation reflects genuine ambiguity in student responses." It could be that it reflects genuine ambiguity on the part of the graders/LLMs as to how a response should be graded.
Earlier quoted context omitted.
My cohort was actively working with invisible realy-inside ear speakers.
I have been wondering if some of my students who demonstrated zero knowledge in class but ace in-class exams were doing something like this. I figured something like a hacked out google glasses would do the trick.
Earlier quoted context omitted.
I basically agree with the thrust of what you're saying, but also: > I wrote assembly and C++ code by hand, and it was expected to compile. No, I never got a chance to try to compile it myself before submitting it for grading. Do you, like, really think this is the best way to assess someone's ability? Can't we find a place between the two extremes? Personally, I'd go with a school-provided computer with a developmen…
The safe middle space still does not involve a computer Lots of my tests involved writing pseudocode, or "Just write something that looks like C or Java". Don't miss the semicolon at the end of the line, but if you write "System.print()" rather than "System.out.printLn()" you might lose a single point. Maybe. If there were specific functions you need to call, it would have a man page or similar on the test itself, or…
> We surveyed students before releasing grades to capture their experience. [...] Only 13% preferred the AI oral format. 57% wanted traditional written exams. [...] 83% of students found the oral exam framework more stressful than a written exam. [...] > Take-home exams are dead. Reverting to pen-and-paper exams in the classroom feels like a regression. Yeah, not sure the conclusion of the article really matches the…
> they expressed a clear preference for written exams When I was a student, I would have been quite vocal with my clear preferences for all exams being open-book and/or being able to amend my answers after grading for a revised score. What I'm saying is, "the students would prefer..." isn't automatically case closed on what's best. Obviously the students would prefer a take-home because you can look up everything you…
How do you expect that to work? After the exam, you talk to your friends (and to ChatGPT) and know the correct answers even if you could have never produced them during the exam.
Earlier quoted context omitted.
Not the comprehensive physics exams I assigned as a prof. A well set exam takes at least 20-30 min to grade. That's 8-12 hours of work, and in practice, took several sittings over several days. If you are going to set an exam that can be graded in 5-10 min, you are not getting a lot of signal out of it. I wanted to do oral exams, but they are much more exhausting for the prof. Nominally, each student is with you for…
> If you are going to set an exam that can be graded in 5-10 min, you are not getting a lot of signal out of it. That's entirely false; this is why we have multiple-choice tests.
- They have a base marks of 20-25% (by random guessing) instead of 0.
- You never see the working. So you can't check if students are thinking correctly. Slightly wrong thinking can get you right answers.
- They don't even remotely reflect real life at all. Written worked through problems on the other hand - I still do those in my professional life as a scientist all the time. It's just that I am setting the questions for myself.
- The format doesn't allow for extended thought questions.
In my undergrad, I had some excellent profs who would set long work through exam question in such a way that you learned something even in the exams. Simply a joy taking those exams that gave a comprehensive walk through of the course. As a prof, I have always tried to replicate that.
This is all so crazy to me. I went to school long before LLMs were even a Google Engineer's brianfart for the transformer paper and the way I took exams was already AI proof. Everything hand written in pen in a proctored gymnasium. No open books. No computers or smart phones, especially ones connected to the internet. Just a department sanctioned calculator for math classes. I wrote assembly and C++ code by hand, and…
You have a very weird idea of education if a teaching method that results in a 99% failure rate is seen as good by yourself. Do you imagine a professional turning out work that was 99% suboptimal?
Earlier quoted context omitted.
At the risk of perhaps stating the obvious, there appears to be a whiff of aggression from this article. The "fighting fire with fire" language, the "haha, we love old FakeFoster, going to have to see if we change that" response to complaints that the voice was intimidating ... if there wasn't a specific desire to punish the class for LLM use by subjecting them to a robotic NKVD interrogation then the authors should…
You can try out the voice yourself. It's not that bad. https://elevenlabs.io/app/talk-to?agent_id=agent_8101k9d1pq4...
Earlier quoted context omitted.
At the risk of perhaps stating the obvious, there appears to be a whiff of aggression from this article. The "fighting fire with fire" language, the "haha, we love old FakeFoster, going to have to see if we change that" response to complaints that the voice was intimidating ... if there wasn't a specific desire to punish the class for LLM use by subjecting them to a robotic NKVD interrogation then the authors should…
You can try out the voice yourself. It's not that bad. https://elevenlabs.io/app/talk-to?agent_id=agent_8101k9d1pq4...
That does not resemble any good professor I've ever heard. It's very aggressive and stern, which is not generally how oral exams are conducted. Feels much more like I'm being cross examined in court.
Earlier quoted context omitted.
> they expressed a clear preference for written exams When I was a student, I would have been quite vocal with my clear preferences for all exams being open-book and/or being able to amend my answers after grading for a revised score. What I'm saying is, "the students would prefer..." isn't automatically case closed on what's best. Obviously the students would prefer a take-home because you can look up everything you…
> being able to amend my answers after grading for a revised score How do you expect that to work? After the exam, you talk to your friends (and to ChatGPT) and know the correct answers even if you could have never produced them during the exam.
This was pre-LLM, but you could cheat back then too. LLMs make it a bit easier by showing you the work to "show" on your corrections.