Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI boosted homework scores, then exam scores dropped: study

economist.com

Post reply on HN
Print view View original

291–300 of 409 posts

Re: AI boosted homework scores, then exam scores dropped: study

#291

Post by ungreased0675 »

asdfman123 wrote: post #279
kzz102 wrote: post #157

Earlier quoted context omitted.

It's not so clear that AI is an amplifier.. The paper has some fascinating analysis on this topic: "At the other end of the distribution, AI students who spend more than 65 minutes on their homework receive homework and exam scores similar to those of non-AI students, suggesting that these students do not use generative AI for homework assignments. However, this group consists entirely of students who adopted generat…

Personally in my own work it's pretty obvious how it's an amplifier. You waste far less time trying to find an answer to something that confuses you. You might say "it's good to learn research skills" and that's true to an extent, but tutors have always made people better students. And AI is a tutor you can message at any time, day or night, for free.

Yes, but is it the correct answer?

I wasted several days trying to have AI teach me containers. I would have been much better off just reading the docs.

Re: AI boosted homework scores, then exam scores dropped: study

#292

Post by stopIiiiit »

kzz102 wrote: post #157
GPerson wrote: post #7

The kids who used AI and studied for a similar amount of time as the non-AI high performers apparently had similar (slightly higher) performance. The kids who used AI and did poorly used the AI to do the homework for them (this is what the article says). I believe AI is basically an amplifier of bad and good. I’m cynical about the world and assume it will be used more for bad than good, but I don’t doubt some of the…

It's not so clear that AI is an amplifier.. The paper has some fascinating analysis on this topic: "At the other end of the distribution, AI students who spend more than 65 minutes on their homework receive homework and exam scores similar to those of non-AI students, suggesting that these students do not use generative AI for homework assignments. However, this group consists entirely of students who adopted generat…

Self reports of AI use is worthless

Social stigma against just juices people to stick to socially desirable answers; no doubt a whole bunch who self reported as non-AI users actually used AI

Re: AI boosted homework scores, then exam scores dropped: study

#293

Post by jasonfarnon »

asdfman123 wrote: post #279
kzz102 wrote: post #157

Earlier quoted context omitted.

It's not so clear that AI is an amplifier.. The paper has some fascinating analysis on this topic: "At the other end of the distribution, AI students who spend more than 65 minutes on their homework receive homework and exam scores similar to those of non-AI students, suggesting that these students do not use generative AI for homework assignments. However, this group consists entirely of students who adopted generat…

Personally in my own work it's pretty obvious how it's an amplifier. You waste far less time trying to find an answer to something that confuses you. You might say "it's good to learn research skills" and that's true to an extent, but tutors have always made people better students. And AI is a tutor you can message at any time, day or night, for free.

"AI is a tutor you can message at any time, day or night, for free"

which makes it not a tutor. if it is true that tutors have always made people better students, then those tutors are definitely imposing some limits on how many answers they give you and requiring you to do some thinking. I think you could have premised your same argument by, "copying a smart kid's answers has always made people better students...."

Re: AI boosted homework scores, then exam scores dropped: study

#294

Post by Mtinie »

ungreased0675 wrote: post #291
asdfman123 wrote: post #279

Earlier quoted context omitted.

Personally in my own work it's pretty obvious how it's an amplifier. You waste far less time trying to find an answer to something that confuses you. You might say "it's good to learn research skills" and that's true to an extent, but tutors have always made people better students. And AI is a tutor you can message at any time, day or night, for free.

Yes, but is it the correct answer? I wasted several days trying to have AI teach me containers. I would have been much better off just reading the docs.

Did you initially ask the tool to read the docs?

I wouldn’t be surprised if you did and still had the same issue, but if you relied on the training data recall alone, you definitely shortchanged yourself.

I’ve had decent results with tasking a model to run pre-research and summarization as a checklist, then have a second instance of the model(s) review and then develop my learning plan learn, versus the times I just asked Claude or ChatGPT to explain something to me “from memory.”

Re: AI boosted homework scores, then exam scores dropped: study

#295

Post by robryan »

jasonfarnon wrote: post #293
asdfman123 wrote: post #279

Earlier quoted context omitted.

Personally in my own work it's pretty obvious how it's an amplifier. You waste far less time trying to find an answer to something that confuses you. You might say "it's good to learn research skills" and that's true to an extent, but tutors have always made people better students. And AI is a tutor you can message at any time, day or night, for free.

"AI is a tutor you can message at any time, day or night, for free" which makes it not a tutor. if it is true that tutors have always made people better students, then those tutors are definitely imposing some limits on how many answers they give you and requiring you to do some thinking. I think you could have premised your same argument by, "copying a smart kid's answers has always made people better students...."

Nothing stopping kids from asking AI to give you responses like this, it is more than capable. There is always going to be the temptation to take short cuts though.

Re: AI boosted homework scores, then exam scores dropped: study

#296

Post by jmcgough »

tehnoslow wrote: post #246

Honestly I wouldn't demonize AI in education. When we were students we were forbidden from using calculators, then the internet and now it's become the norm. It's the same with AI, I think. AI has already become a part of our lives. The important thing is just teaching children how to use it correctly.

Unlike AI, calculator use has been studied and shown to be useful for math education. Our understanding of these things evolved over time, however the results of this study are concerning.

Re: AI boosted homework scores, then exam scores dropped: study

#297

Post by asdfman123 »

jasonfarnon wrote: post #293
asdfman123 wrote: post #279

Earlier quoted context omitted.

Personally in my own work it's pretty obvious how it's an amplifier. You waste far less time trying to find an answer to something that confuses you. You might say "it's good to learn research skills" and that's true to an extent, but tutors have always made people better students. And AI is a tutor you can message at any time, day or night, for free.

"AI is a tutor you can message at any time, day or night, for free" which makes it not a tutor. if it is true that tutors have always made people better students, then those tutors are definitely imposing some limits on how many answers they give you and requiring you to do some thinking. I think you could have premised your same argument by, "copying a smart kid's answers has always made people better students...."

That's why you see the split in outcomes. The fraction of kids who value self improvement are going to get smarter, and the ones who just want to slide by will fall behind.

Re: AI boosted homework scores, then exam scores dropped: study

#298

Post by bsder »

the_real_cher wrote: post #11
GPerson wrote: post #7

The kids who used AI and studied for a similar amount of time as the non-AI high performers apparently had similar (slightly higher) performance. The kids who used AI and did poorly used the AI to do the homework for them (this is what the article says). I believe AI is basically an amplifier of bad and good. I’m cynical about the world and assume it will be used more for bad than good, but I don’t doubt some of the…

This is my thoughts as well! It makes smarter people smarter and dumb people dumber!

AI does not make me "smarter" in the important thing I am working on.

In software particularly, what AI does do is give me back my time from the drudge work that I don't care about. Keeping my build system configured and my tests up to date and my documentation synchronized is a good use of AI because I mostly don't care about how crummy the result is as long as it "works".

Re: AI boosted homework scores, then exam scores dropped: study

#299

Post by 1saadcodes »

AI itself is not the problem imo. The core issue is basically outsourcing your entire brain to the LLM and never trying to solve a problem yourself

When I was in school there were multiple instances where I would be stuck on a problem for nearly an hr but I always learnt something from it. They key I think is to accurately identify when to and when not to use AI

Re: AI boosted homework scores, then exam scores dropped: study

#300

Post by myhf »

stopIiiiit wrote: post #292
kzz102 wrote: post #157

Earlier quoted context omitted.

It's not so clear that AI is an amplifier.. The paper has some fascinating analysis on this topic: "At the other end of the distribution, AI students who spend more than 65 minutes on their homework receive homework and exam scores similar to those of non-AI students, suggesting that these students do not use generative AI for homework assignments. However, this group consists entirely of students who adopted generat…

Self reports of AI use is worthless Social stigma against just juices people to stick to socially desirable answers; no doubt a whole bunch who self reported as non-AI users actually used AI

Sounds like the opposite of the social stigma in tech companies, where most people are under pressure to self-report as AI users when they are actually just using normal methods to get the same result (and spending the time saved however they like) (nothing against genAI for making work take more time, it's still good for business as long as the extra work can be moved around)
Post reply on HN