Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI boosted homework scores, then exam scores dropped: study

economist.com

Post reply on HN
Print view View original

221–230 of 409 posts

Re: AI boosted homework scores, then exam scores dropped: study

#221

Post by kzz102 »

bonoboTP wrote: post #210
kzz102 wrote: post #157

Earlier quoted context omitted.

It's not so clear that AI is an amplifier.. The paper has some fascinating analysis on this topic: "At the other end of the distribution, AI students who spend more than 65 minutes on their homework receive homework and exam scores similar to those of non-AI students, suggesting that these students do not use generative AI for homework assignments. However, this group consists entirely of students who adopted generat…

It would be good to see the effect of access to AI during preparation on those who previously achieved top 10% points in exams of similar topics before. I suspect they would benefit further.

My apologies for coming off as over-enthusiastic, I am currently obsessed with this study. Here is another quote:

"The negative learning effects are larger for students with higher initial achievement. The differences in the estimated full (6-10 month average) effects are substantial, with a 50% gap between the most negative effect (-24 percent) for the highest tercile and the least negative (-16 percent) for the lowest tercile. "

Not top 10% as you asked, but the closest to what you asked. My working hypothesis is that top performance is highly correlated with willingness to work hard, and AI decreases the motivation to work hard.

Re: AI boosted homework scores, then exam scores dropped: study

#222

Post by criddell »

ekjhgkejhgk wrote: post #211
criddell wrote: post #87

Earlier quoted context omitted.

This was probably middle school.

What do you think about it? I think that the ability to study by yourself is probably the most valuable thing one could get from school and most kids usually don't. Your setup would suggest that demanding that a kids know how to study by themselves is going mainstream...? Your answer - middle school - I find it extremely young for this, which is extra interesting.

IMHO, my kid was assigned way too much homework. One year I eventually told her to do some amount per class (I think it was 45 minutes) and then be done with it so that she would have time for sports and family stuff. She was spending 4 or 5 hours every night on homework and this was in 4th grade.

That year I sent an email to her teacher to let them know that I'm the one responsible for her not doing all the work and if that's a problem, we need to talk. It wasn't a problem.

The one class that was flipped was a relief because she could breeze through the lesson much faster than would normally be spent on it in class and the amount of time on exercises was limited to the class time slot.

Re: AI boosted homework scores, then exam scores dropped: study

#224

Post by bonoboTP »

eastbound wrote: post #191
WalterBright wrote: post #177

Earlier quoted context omitted.

> Homework should be optional material for self study At Caltech, homework was assigned but had no bearing on your grade. The grades were based on the midterm and final exams. But not mastering the homework usually resulted in flunking the exams. There were "retch" sessions after each homework assignment that was staffed by a grad student, and the purpose was to help the students understand the homework problems. I k…

> 1. never miss a lecture, no matter what As a student of a top-tier French Master's degree, I consulted with a teacher to deal with exhaustion and to ask for a class rescheduling for my case. Explaining my situation, the teacher looks at me and interjected: —Waitwaitwait. You... you went to all lectures!? And he was right. Rather than following the curriculum, I should have developed my taste for various engineering…

I think the first semester, or at least half of that, you should go to every lecture until you develop taste for what's useful. By the time you do your master's you should have a good sense for what classes are serious and have useful lectures and where you can just cram last minute because it's a bullshit class and where you're better off studying from a textbook.

Re: AI boosted homework scores, then exam scores dropped: study

#225

Post by jonahx »

kzz102 wrote: post #157
GPerson wrote: post #7

The kids who used AI and studied for a similar amount of time as the non-AI high performers apparently had similar (slightly higher) performance. The kids who used AI and did poorly used the AI to do the homework for them (this is what the article says). I believe AI is basically an amplifier of bad and good. I’m cynical about the world and assume it will be used more for bad than good, but I don’t doubt some of the…

It's not so clear that AI is an amplifier.. The paper has some fascinating analysis on this topic: "At the other end of the distribution, AI students who spend more than 65 minutes on their homework receive homework and exam scores similar to those of non-AI students, suggesting that these students do not use generative AI for homework assignments. However, this group consists entirely of students who adopted generat…

Depends who's using it. Like many tools the force multiplier depends on the operator.

I'm confident it's an amplifier for people who know how learning works and already do a lot of it, successfully. However the level of "learning fluency" I'm talking about isn't reached for many until late college or grad school, and sometimes not at all. So I'm not surprised by the quoted results for 12-18 year olds.

Re: AI boosted homework scores, then exam scores dropped: study

#226

Post by frollogaston »

GPerson wrote: post #206
frollogaston wrote: post #141

Earlier quoted context omitted.

I don't think this is true, and frankly, I don't care what they do in Europe.

It is what all my European peers reported to me during grad school, and you should care because it is an example of a system which solves this problem successfully (at the university level).

Oh, university level? The article is about 12-18yo. American grad school already doesn't have graded homework, and undergrad sometimes does or sometimes not.

Re: AI boosted homework scores, then exam scores dropped: study

#227

Post by ACow_Adonis »

grey-area wrote: post #156
the_real_cher wrote: post #11

Earlier quoted context omitted.

This is my thoughts as well! It makes smarter people smarter and dumb people dumber!

There is zero evidence for this. LLMs make significant mistakes frequently and smart people have no way of judging those mistakes outside their domain expertise. They are also sycophantic and great at being an echo chamber which makes people feel smart even if they are not. So I think the burden of proof is on you to prove that they somehow amplify intelligence, it seems highly unlikely.

Why would a smart person go to an LLM for an answer they cannot judge or test, be succeptible to flattery and sycophancy rather than picking up on the emotional manipulation and being suspicious/sceptical of the interaction, or looking for support from an echo chamber rather than a Socratic opponent?

All of those sound like flaws and defects of dumb people?

Re: AI boosted homework scores, then exam scores dropped: study

#228

Post by bonoboTP »

kzz102 wrote: post #221
bonoboTP wrote: post #210

Earlier quoted context omitted.

It would be good to see the effect of access to AI during preparation on those who previously achieved top 10% points in exams of similar topics before. I suspect they would benefit further.

My apologies for coming off as over-enthusiastic, I am currently obsessed with this study. Here is another quote: "The negative learning effects are larger for students with higher initial achievement. The differences in the estimated full (6-10 month average) effects are substantial, with a 50% gap between the most negative effect (-24 percent) for the highest tercile and the least negative (-16 percent) for the low…

Interesting. Top academic performance is mostly correlated with conscientiousness (willingness to work hard and keeping track of things) and intelligence. And I'd add motivation and interest to that too.

I think if you take a physics class where the student is intelligent and intrinsically motivated through their own interest (I admit this is rare) then AI probably helps.

Re: AI boosted homework scores, then exam scores dropped: study

#229

Post by heddhunter »

dataflow wrote: post #215
moffkalast wrote: post #196

Earlier quoted context omitted.

I thought that it was generally accepted by now that homework in the volumes that it is being assigned in the modern day was not found to be beneficial in any significant way in the first place? Maybe once all students start outsourcing it to AI it might finally die like it deserves to. People these days grow up with almost no free time for themselves, it's all school, sports/extracurriculars or homework nonstop. All…

> I thought that it was generally accepted by now that homework in the volumes that it is being assigned in the modern day was not found to be beneficial in any significant way in the first place? Citation needed? I have no clue where you got this from. I hadn't even heard of it as a conjecture, let alone as something anyone accepted, let alone as generally accepted...

https://www.tandfonline.com/doi/pdf/10.1080/00220973.2012.74...

Re: AI boosted homework scores, then exam scores dropped: study

#230

Post by nostrademons »

ezst wrote: post #179
nostrademons wrote: post #133

Earlier quoted context omitted.

This is typical at the college/university level - when I attended 20-25 years ago, we were expected to learn the material through readings outside of class, and then class time was spent on discussion, application, questions, and working through problems. IMHO there's a lot to recommend this style. A lot of students learn better when they can wrestle with the material at their own pace, on their own time, in their ow…

Although obvious, I think it's worth stating that this method mostly (only?) makes sense for higher education, where students are conscious, involved and engaged in their curriculum.

I think it makes sense as soon as students have the fundamentals down. You have to be able to read, listen, make connections, think critically, devote sustained attention to a task, and manage your time effectively. Some students develop these skills in elementary school, and then starting in middle school, the flipped classroom approach can work. Some adults never develop these skills. Although arguably, people that never develop these skills will likely be sitting there zoning out in a traditional classroom lecture and not actually learn anything.
Post reply on HN