Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI boosted homework scores, then exam scores dropped: study

economist.com

Post reply on HN
Print view View original

231–240 of 409 posts

Re: AI boosted homework scores, then exam scores dropped: study

#231

Post by sensanaty »

magicalhippo wrote: post #183
nxc18 wrote: post #162

Earlier quoted context omitted.

> help. It could explain concepts in a way that might help you understand better People say that all the time, but does anyone really think a lack of good explanations for things is a limiting factor in 2026? Or even 2010?

Back when I was studying there were topics which I had a really hard time grasping. Having a personal tutor that I could ask specific questions and have a back and forth with would have helped a lot I think. At least the few times I did have such an opportunity that was definitely the case.

Kids are, by and large, lazy however. Hell, adults are lazy, this isn't even an indictment on kids or even people in general, I think our bodies & brains are simply wired to seek the path of least resistance from an evolutionary POV.

Kids won't be using LLM tooling as a personal tutor following some sort of Socratic method, they'll ask it to solve their home/coursework for them and blindly copy/paste the answer. Hell, they'll just manually copy down what's on their screen if copy/pasting isn't possible for whatever reason.

Obviously exceptions exist, but I'd wager from being an ex-kid myself the type of kid who would genuinely use these tools for actual proper self-tutoring would be an extreme rarity.

Re: AI boosted homework scores, then exam scores dropped: study

#232

Post by jknoepfler »

GPerson wrote: post #7

The kids who used AI and studied for a similar amount of time as the non-AI high performers apparently had similar (slightly higher) performance. The kids who used AI and did poorly used the AI to do the homework for them (this is what the article says). I believe AI is basically an amplifier of bad and good. I’m cynical about the world and assume it will be used more for bad than good, but I don’t doubt some of the…

It's a familiar story. If you copy/paste Wikipedia and turn it in, you learn nothing. If you skim Wikipedia, hit up the references, the consult other sources, then synthesize your own thought, you probably get farther faster than you would without it.

We could probably cook up thousands of examples of the same problem (YouTube DIY tutorials, GPS navigation, etc.)

Re: AI boosted homework scores, then exam scores dropped: study

#233

Post by 0x457 »

ezst wrote: post #201
0x457 wrote: post #72

Earlier quoted context omitted.

Yup, when you find such exam please let us know.

I had the privilege of being educated in several countries and cultures. This might come across as "your education system sucks", and that's essentially what it boils down to, but on the upside I'm truly sorry for you and will oppose a comment to your cynicism. Good exams test your ability to express coherent thoughts about the concepts you learned, and let you generalize upon them by having you use and combine them…

Same, but all the exams I ever took mostly tested your memory more than anything else. I don't think it's possible to test people at scale any other way.

Re: AI boosted homework scores, then exam scores dropped: study

#234

Post by throwyawayyyy »

GPerson wrote: post #7

The kids who used AI and studied for a similar amount of time as the non-AI high performers apparently had similar (slightly higher) performance. The kids who used AI and did poorly used the AI to do the homework for them (this is what the article says). I believe AI is basically an amplifier of bad and good. I’m cynical about the world and assume it will be used more for bad than good, but I don’t doubt some of the…

This is kind of damning, isn't it? "Similar or slightly higher" for spending the _same time_ means it's not, in fact, helping. At the same time as being extremely damaging to a significant number of students who, of course, use the AI to cheat. Because everything about it makes cheating easy.

Re: AI boosted homework scores, then exam scores dropped: study

#235

Post by insane_dreamer »

a2ff6eeb0 wrote: post #14

[flagged]

We could also drop school altogether and just make sure every kid has an LLM-connected device that's voice driven. They wouldn't even need to learn to read and write -- that's old school.

update: this would make universal education free! because MegaCorp would happily give/support the devices free of charge (in exchange for ads being inserted into the responses)

Re: AI boosted homework scores, then exam scores dropped: study

#236

Post by bawolff »

jonahx wrote: post #225
kzz102 wrote: post #157

Earlier quoted context omitted.

It's not so clear that AI is an amplifier.. The paper has some fascinating analysis on this topic: "At the other end of the distribution, AI students who spend more than 65 minutes on their homework receive homework and exam scores similar to those of non-AI students, suggesting that these students do not use generative AI for homework assignments. However, this group consists entirely of students who adopted generat…

Depends who's using it. Like many tools the force multiplier depends on the operator. I'm confident it's an amplifier for people who know how learning works and already do a lot of it, successfully. However the level of "learning fluency" I'm talking about isn't reached for many until late college or grad school, and sometimes not at all. So I'm not surprised by the quoted results for 12-18 year olds.

And people who want to learn. Most teenagers lack agency in their studies. They aren't in high school because they love it but because they have no choice.

Re: AI boosted homework scores, then exam scores dropped: study

#237

Post by ChrisSD »

jonahx wrote: post #225
kzz102 wrote: post #157

Earlier quoted context omitted.

It's not so clear that AI is an amplifier.. The paper has some fascinating analysis on this topic: "At the other end of the distribution, AI students who spend more than 65 minutes on their homework receive homework and exam scores similar to those of non-AI students, suggesting that these students do not use generative AI for homework assignments. However, this group consists entirely of students who adopted generat…

Depends who's using it. Like many tools the force multiplier depends on the operator. I'm confident it's an amplifier for people who know how learning works and already do a lot of it, successfully. However the level of "learning fluency" I'm talking about isn't reached for many until late college or grad school, and sometimes not at all. So I'm not surprised by the quoted results for 12-18 year olds.

Sometimes things are just common sense pure and simple.

Re: AI boosted homework scores, then exam scores dropped: study

#238

Post by baud9600 »

I am waiting for AI Viva Voce (“AVV”). I am really surprised we aren’t hearing more about this idea?

Try it for yourself by making a prompt like this (adapt as needed):

Pretend I am an undergraduate student of Computer Science. I am learning about early microprocessors from the 1970s. I want you to ask me an examination question as if you were doing a viva voce exam with me, to test my understanding of concepts. I want you to receive my answer and then based on what I said I want you to ask me a more specific question to probe my understanding. Repeat this interaction up to 5 times. Then grade my understanding so far, by giving me a pass, merit, credit, or distinction. Can you explain how you arrive at the grade based on my answers and your expectation of undergraduate knowledge of microprocessor theory?

Re: AI boosted homework scores, then exam scores dropped: study

#239

Post by jvvw »

WalterBright wrote: post #177
tancop wrote: post #163

All AI did here is exposed an old problem in education. Kids are expected to get everything perfectly for the first time but a lot of them don't so the class gets easier next year to keep up pass rates. We need to restructure the system to treat failure as a signal instead of a disaster. Grades should come from hard randomized exams with unlimited retakes so one bad day won't hurt you. Homework should be optional mat…

> Homework should be optional material for self study At Caltech, homework was assigned but had no bearing on your grade. The grades were based on the midterm and final exams. But not mastering the homework usually resulted in flunking the exams. There were "retch" sessions after each homework assignment that was staffed by a grad student, and the purpose was to help the students understand the homework problems. I k…

Likewise at Oxford - only the final exams counted (though you had to pass first year exams to continue to the second year).

But you had tutorials each week, and if your tutors thought you weren't doing enough work they could set you exams mid-course called 'penal collections' and if you failed them you could be thrown out. They were rare but definitely not unknown.

Re: AI boosted homework scores, then exam scores dropped: study

#240

Post by dan_ggggg »

ChrisSD wrote: post #237
jonahx wrote: post #225

Earlier quoted context omitted.

Depends who's using it. Like many tools the force multiplier depends on the operator. I'm confident it's an amplifier for people who know how learning works and already do a lot of it, successfully. However the level of "learning fluency" I'm talking about isn't reached for many until late college or grad school, and sometimes not at all. So I'm not surprised by the quoted results for 12-18 year olds.

Sometimes things are just common sense pure and simple.

> Sometimes things are just common sense pure and simple.

The discussion is about "AI", so common sense is out the window. These people's professional reputations depend on addict-level "AI" usage remaining socially acceptable.

Post reply on HN