Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI boosted homework scores, then exam scores dropped: study

economist.com

Post reply on HN
Print view View original

201–210 of 409 posts

Re: AI boosted homework scores, then exam scores dropped: study

#201

Post by ezst »

0x457 wrote: post #72
rektomatic wrote: post #36

Earlier quoted context omitted.

Good exams don't just test memorization

Yup, when you find such exam please let us know.

I had the privilege of being educated in several countries and cultures. This might come across as "your education system sucks", and that's essentially what it boils down to, but on the upside I'm truly sorry for you and will oppose a comment to your cynicism. Good exams test your ability to express coherent thoughts about the concepts you learned, and let you generalize upon them by having you use and combine them to solve cases and problems that were not those seen in the teaching material. You can even teach and test history that way, and make it much more insightful and captivating than a laundry list of dates and events (as I suppose was your case?)

Re: AI boosted homework scores, then exam scores dropped: study

#202

Post by cheesecakegood »

yoyohello13 wrote: post #88
criddell wrote: post #65

For some of the classes my kids were in, the lessons and homework time were reversed. Learning the material (using the text book and videos) was what they had to do at night and class time was spent working through problems or applying the material in some way.

Flipped classroom was a popular method when I was teaching math. Some students found it very helpful. The trouble was getting the students to do the reading before class. If students came in unprepared they struggled to keep up.

Popular yes; based on good quality research… no, not really. It’s really pretty unfortunate timing with the AI boom, because the field of education research is not very high on the science quality totem pole. And the field as a whole was and is also slow to incorporate neuroscience findings, too.

Re: AI boosted homework scores, then exam scores dropped: study

#203

Post by ambicapter »

GPerson wrote: post #7

The kids who used AI and studied for a similar amount of time as the non-AI high performers apparently had similar (slightly higher) performance. The kids who used AI and did poorly used the AI to do the homework for them (this is what the article says). I believe AI is basically an amplifier of bad and good. I’m cynical about the world and assume it will be used more for bad than good, but I don’t doubt some of the…

> I believe AI is basically an amplifier of bad and good.

Same can be said of technology in general tbh.

Re: AI boosted homework scores, then exam scores dropped: study

#205

Post by moffkalast »

WalterBright wrote: post #177
tancop wrote: post #163

All AI did here is exposed an old problem in education. Kids are expected to get everything perfectly for the first time but a lot of them don't so the class gets easier next year to keep up pass rates. We need to restructure the system to treat failure as a signal instead of a disaster. Grades should come from hard randomized exams with unlimited retakes so one bad day won't hurt you. Homework should be optional mat…

> Homework should be optional material for self study At Caltech, homework was assigned but had no bearing on your grade. The grades were based on the midterm and final exams. But not mastering the homework usually resulted in flunking the exams. There were "retch" sessions after each homework assignment that was staffed by a grad student, and the purpose was to help the students understand the homework problems. I k…

That only works with competent lecturers. I can speak for my experience at a more mid university, where that would apply to about two thirds of the classes. For the remaining third, going to the lectures felt genuinely counterproductive and you could actually feel yourself losing braincells listening to the confused nonsense or classes held in English for Erasmus students by someone who could barely speak it coherently. It was just a complete waste of already little available time.

So the endgame was figuring out what the tests in previous years looked like (cause it was likely gonna be a copy paste affair), do a targeted study run for those exercises and 9/10 you would pass.

Re: AI boosted homework scores, then exam scores dropped: study

#206

Post by GPerson »

frollogaston wrote: post #141
GPerson wrote: post #29

Earlier quoted context omitted.

This is the answer. Many (all?) European schools don’t give homework (this is partly because of the way the grad programs work). It’s on the students to study in order to pass the (difficult) exams.

I don't think this is true, and frankly, I don't care what they do in Europe.

It is what all my European peers reported to me during grad school, and you should care because it is an example of a system which solves this problem successfully (at the university level).

Re: AI boosted homework scores, then exam scores dropped: study

#207

Post by thaumasiotes »

doctorpangloss wrote: post #197
kzz102 wrote: post #157

Earlier quoted context omitted.

It's not so clear that AI is an amplifier.. The paper has some fascinating analysis on this topic: "At the other end of the distribution, AI students who spend more than 65 minutes on their homework receive homework and exam scores similar to those of non-AI students, suggesting that these students do not use generative AI for homework assignments. However, this group consists entirely of students who adopted generat…

well most university exams are designed to measure how much you study. so we didn't really need a study to tell us, "Exams continue to measure what they are designed to measure." they're not designed to measure general aptitude, or function as admissions criteria, or screen for job applications, or any other numerous things they are used for. there can be many questions of pedagogy. one of them is, what do our exams…

> well most university exams are designed to measure how much you study.

Huh? They're designed to measure how much you know. They can't see how much you study, nor would they have reason to be interested.

Re: AI boosted homework scores, then exam scores dropped: study

#208

Post by idiotsecant »

moffkalast wrote: post #196
nerevarthelame wrote: post #149

Earlier quoted context omitted.

The scary thing is that 81% of AI users in this study were determined to be "outsourcing" their homework to the LLMs - and the rate increases the more exposure they had to LLMs. The "slightly higher" performance is based on statistically insignificant samples (between 4 and 20 students, depending on the context, out of the total population of 26,000): https://bsky.app/profile/benjaminjriley.bsky.social/post/3mt...

I thought that it was generally accepted by now that homework in the volumes that it is being assigned in the modern day was not found to be beneficial in any significant way in the first place? Maybe once all students start outsourcing it to AI it might finally die like it deserves to. People these days grow up with almost no free time for themselves, it's all school, sports/extracurriculars or homework nonstop. All…

All play and no work makes for an even more dysfunctional society. Kids need to do their homework, both to train their minds but also to develop integrity and work ethic. Being able to diligently work toward a distant goal is not something you're born with.

Re: AI boosted homework scores, then exam scores dropped: study

#209

Post by doctorpangloss »

thaumasiotes wrote: post #207
doctorpangloss wrote: post #197

Earlier quoted context omitted.

well most university exams are designed to measure how much you study. so we didn't really need a study to tell us, "Exams continue to measure what they are designed to measure." they're not designed to measure general aptitude, or function as admissions criteria, or screen for job applications, or any other numerous things they are used for. there can be many questions of pedagogy. one of them is, what do our exams…

> well most university exams are designed to measure how much you study. Huh? They're designed to measure how much you know. They can't see how much you study, nor would they have reason to be interested.

at least in my experience in university - i didn't really ask this question, since it is obvious to me, but some students have asked it during lecture, or some instructors have volunteered the answer ahead of time - if you ask how to perform better on the exam, usually the instructors say, "here's what you should study." they never say, "know more." the thing i am talking about is consistent with the paper. really, your takeaway should be, exams can't see how much you know!

Re: AI boosted homework scores, then exam scores dropped: study

#210

Post by bonoboTP »

kzz102 wrote: post #157
GPerson wrote: post #7

The kids who used AI and studied for a similar amount of time as the non-AI high performers apparently had similar (slightly higher) performance. The kids who used AI and did poorly used the AI to do the homework for them (this is what the article says). I believe AI is basically an amplifier of bad and good. I’m cynical about the world and assume it will be used more for bad than good, but I don’t doubt some of the…

It's not so clear that AI is an amplifier.. The paper has some fascinating analysis on this topic: "At the other end of the distribution, AI students who spend more than 65 minutes on their homework receive homework and exam scores similar to those of non-AI students, suggesting that these students do not use generative AI for homework assignments. However, this group consists entirely of students who adopted generat…

It would be good to see the effect of access to AI during preparation on those who previously achieved top 10% points in exams of similar topics before. I suspect they would benefit further.
Post reply on HN