Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kids who use ChatGPT as a study assistant do worse on tests

hechingerreport.org

Post reply on HN
Print view View original

61–70 of 168 posts

Re: Kids who use ChatGPT as a study assistant do worse on tests

#61

Post by dailykoder »

alabhyajindal wrote: post #24

I recently used AI assistants for help with programming homework. My usual prompts include "help me think in the right direction", "is my thinking correct" etc. I also find myself copy pasting a question in chat to understand it better. I had the suspicion that this is not aiding in my learning process even though I am able to "solve" more problems. Nice to see this confirmed. Time to stop!

I learned programming before genAI, but with plenty of video tutorials and stackoverflow available.

Someday I noticed that I would automatically hit up the search engine within seconds when I hit a road block. It's kinda the same as with GenAI. You just have to know how to query search engines efficiently. I got stuff done, but I often did not learn much and sometimes didn't even understand what I was doing. So I started to force myself to use data- or reference sheets and try to come up with a solution myself. And if that wasn't good enough or I had real trouble, then I tried using some different resource. And it improved my programming experience tremendously. It's hard, but it's worth it.

Re: Kids who use ChatGPT as a study assistant do worse on tests

#62

Post by langsoul-com »

Not surprising. Test is all about memorising things. If you don't need to memorise everything because it's on google, you won't.

Thus, when the test rolls around, nothing is memorised and then they do bad.

It's like memorising phone numbers VS keeping in the contacts app. Before I memorised tons of numbers, but now they're all on the app and I barely recall my own.

Re: Kids who use ChatGPT as a study assistant do worse on tests

#63

Post by cageface »

eks391 wrote: post #60

If I lived before the tape measure was invented, and rely on carefully placing my metersticks to measure things, I can get really good at measuring without the need for a measuring tape. After all, a measuring tape is just a few flexible metersticks anyways, so if you need to measure something longer than the full length of the tape, you are screwed. If you take the measuring tape away from the person who relied on t…

What if the function they were using the tool for is basic reasoning?

Re: Kids who use ChatGPT as a study assistant do worse on tests

#64

Post by LeonB »

ShadowBlades512 wrote: post #34
lll-o-lll wrote: post #5

When I was young, and learning math, my father always forbade me from looking at the answer in the back of the textbook. “You don’t work backwards from the answer!”, and I think this is right. In life, we rarely have the answer in front of us, we have to work that out from the things we know. It’s this struggling that builds a muscle you can then apply to any problem. ChatGPT, I suspect, is akin to looking up the ans…

I think this is very correct for studying, when I was in undergrad, if I saw the answer, I found it was more effective to skip past it and solve it later. Even more important, once I am done, I should not go look at the answer to confirm if I am right or wrong, I should try and validate the answer by looking at my solution and trying to figure out if my solution is correct or not because there are plenty of ways to "…

Good point about disproving your answer (before seeing the alleged real answer). The ability to quickly verify/disprove the answer is a separate and useful skill all by itself, that isn’t explicitly taught.

Troubleshooting/problem solving, working backwards from symptom to problem etc — that’s another, richer and very rewarding “real world” type of skill that isn’t directly taught, (though often encountered incidentally).

Re: Kids who use ChatGPT as a study assistant do worse on tests

#65

Post by notatoad »

lll-o-lll wrote: post #5

When I was young, and learning math, my father always forbade me from looking at the answer in the back of the textbook. “You don’t work backwards from the answer!”, and I think this is right. In life, we rarely have the answer in front of us, we have to work that out from the things we know. It’s this struggling that builds a muscle you can then apply to any problem. ChatGPT, I suspect, is akin to looking up the ans…

>In life, we rarely have the answer in front of us,

this sounds a lot like the "when you're an adult, you won't always have a calculator in your pocket" line we all heard in elementary school. and of course, we all know now how wrong that was.

in life, chatGPT exists. and tools like it are only going to become more widespread. we are going to have the answers in front of us. knowing the things an LLM knows is not a useful skill anymore.

Re: Kids who use ChatGPT as a study assistant do worse on tests

#66

Post by madhatter999 »

Jtsummers wrote: post #53
madhatter999 wrote: post #22

From the abstract: “Consistent with prior work, our results show that access to GPT-4 significantly improves performance (48% improvement for GPT Base and 127% for GPT Tutor). However, we additionally find that when access is subsequently taken away, students actually perform worse than those who never had access (17% reduction for GPT Base).” Kids who use ChatGPT do actually “significantly” better according to the a…

> Also “access to GPT“ in this case is having GPT open while solving the problems, not studying with GPT and then taking the test If this is your takeaway you misread the paper. Students have access to GPT (if they have access, the control didn't) while working through practice problems. Not for the exam itself. From the paper in the experimental design section: > Each session has three parts: > 1. In the first part,…

Thanks for the clarification. I did look at it pretty quickly initially. Students might be over-relying on GPTs, which means less studying, which means less useful retention in the exam

Re: Kids who use ChatGPT as a study assistant do worse on tests

#67

Post by arrowsmith »

JumpCrisscross wrote: post #55
dghlsakjg wrote: post #44

Used incorrectly. Yes. LLMs, for me, have been tremendously useful in learning new concepts. I frequently feed it my own notes and ask it to correct any misunderstandings, or to expand on things I don’t understand. I use it like I would an on demand tutor, but I can totally understand how it could be used as a shortcut that wouldn’t be helpful. In the same way, I can hire a tutor that will help me actually learn, or…

> frequently feed it my own notes and ask it to correct any misunderstandings, or to expand on things I don’t understand In what fields? I’ve tried this with some simple finance and aerospace problems; it’s me, sophomore year undergrad, except someone laced everything I drank with LSD.

I find it's good at answering my questions about humanities subjects e.g. history.

Maybe it's only as good as an undergraduate, but I don't have a humanities degree, so an undergraduate is someone I can learn from.

Re: Kids who use ChatGPT as a study assistant do worse on tests

#68

Post by JumpCrisscross »

arrowsmith wrote: post #67
JumpCrisscross wrote: post #55

Earlier quoted context omitted.

> frequently feed it my own notes and ask it to correct any misunderstandings, or to expand on things I don’t understand In what fields? I’ve tried this with some simple finance and aerospace problems; it’s me, sophomore year undergrad, except someone laced everything I drank with LSD.

I find it's good at answering my questions about humanities subjects e.g. history. Maybe it's only as good as an undergraduate, but I don't have a humanities degree, so an undergraduate is someone I can learn from.

If it’s leading you to sources, go for it. If you’re taking it point blank, consider what you’re learning. (If it isn’t pointing you to sources, ask for them.)

Re: Kids who use ChatGPT as a study assistant do worse on tests

#69

Post by JTyQZSnP3cQGa8B »

gitroom wrote: post #11

What were the primary reasons that made students who used ChatGPT do poorly on math assessments, even though they had worked correctly through a greater number of practice problems?

> they had worked correctly

ChatGPT worked correctly for them but they learned nothing. It should be pretty obvious that you don’t learn by copying answers.

Re: Kids who use ChatGPT as a study assistant do worse on tests

#70

Post by onehair »

alabhyajindal wrote: post #24

I recently used AI assistants for help with programming homework. My usual prompts include "help me think in the right direction", "is my thinking correct" etc. I also find myself copy pasting a question in chat to understand it better. I had the suspicion that this is not aiding in my learning process even though I am able to "solve" more problems. Nice to see this confirmed. Time to stop!

> I also find myself copy pasting a question in chat to understand it better.

This is different fromt students relying on ChatGPT to pass tests.

Your use is much more adulty and you're trying to understand before proceeding. You use ChatGPT like a tutor rather than a calculator, which is improving what you know rather than taking from it

Post reply on HN