Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kids who use ChatGPT as a study assistant do worse on tests

hechingerreport.org

Post reply on HN
Print view View original

111–120 of 168 posts

Re: Kids who use ChatGPT as a study assistant do worse on tests

#111

Post by godelski »

ddfs123 wrote: post #87
notatoad wrote: post #65

Earlier quoted context omitted.

>In life, we rarely have the answer in front of us, this sounds a lot like the "when you're an adult, you won't always have a calculator in your pocket" line we all heard in elementary school. and of course, we all know now how wrong that was. in life, chatGPT exists. and tools like it are only going to become more widespread. we are going to have the answers in front of us. knowing the things an LLM knows is not a u…

>we all know now how wrong that was. How wrong was it ? I find myself having to do head math every time I playing video games.

I do mental math pretty much every day. Even just going through the grocery store.

I also use things I learned from calculus and physics. Like when driving. No, not solving integrals all the time but it does teach you to focus on rates of change

Re: Kids who use ChatGPT as a study assistant do worse on tests

#112

Post by hilbert42 »

lll-o-lll wrote: post #5

When I was young, and learning math, my father always forbade me from looking at the answer in the back of the textbook. “You don’t work backwards from the answer!”, and I think this is right. In life, we rarely have the answer in front of us, we have to work that out from the things we know. It’s this struggling that builds a muscle you can then apply to any problem. ChatGPT, I suspect, is akin to looking up the ans…

"It’s this struggling that builds a muscle…"

Very few of us have the luxury or good fortune to achieve anything of substance without effort, as the old saying goes 'no pain, no gain'.

Good teachers help and are essential for encouragement. If or when AI becomes a truly encouraging mentor then it will be useful.

When a kid I recall hearing a recording of Sparky's Magic Piano on the radio which drove home that there's no shortcut and that effort and hard work eventually pay off. For anyone that hasn't heard it there's a copy on the Internet Archive: https://archive.org/details/78_sparkys-magic-piano_henry-bla...

Here's a Wiki synopsis: https://en.m.wikipedia.org/wiki/Sparky%27s_Magic_Piano

Re: Kids who use ChatGPT as a study assistant do worse on tests

#113

Post by jgb1984 »

I have a visceral dislike, even hate, for what the LLM hype brought the world. The never ending slop it is spouting, filling up the entire internet. More and more I get confronted with images and media that turn out to be AI generated, when I find out I am disgusted and just close the tab.

Soulless drivel, endlessly streaming.

And I'm confident that the education system as we know it will be severely damaged because of it.

Even in our own field, I can guarantee you that software developers that "grew up" with these garbage AI assistants will be worse coders than the generation that came before. You will never develop the understanding, the insight, that's needed by chatgpt'ing your way through college and life.

Excellent news for my own market value of course, but I don't hesitate to say that I regret the LLM hype happened, the impact on the world is overwhelmingly negative (not even touching on the catastrophic environmental and financial cost to society).

Re: Kids who use ChatGPT as a study assistant do worse on tests

#114

Post by lyu07282 »

tripletao wrote: post #84

These comments are filled with misunderstandings of the result. There were three groups of kids: 1. Control, with no LLM assistance at any time. 2. "GPT Base", raw ChatGPT as provided by OpenAI. 3. "GPT Tutor", improved by the researchers to provide hints rather than complete answers and to make fewer mistakes on their specific problems. On study problem sets ("as a study assistant"), kids with access to either GPT d…

Reading comprehension is really awful nowadays or people tend to just comprehend what confirms their prior beliefs. The sad part is that none of those people will ever realize the errors in their comprehension of the article. That's exactly one of the mechanisms how people form wrong opinions, it's compounding and almost impossible to change.

The takeaway should be (besides the research still needing reproduction) to encourage the control of the type of AI agent that is given to students, ones that don't just give answers to copy but provide tutoring. OpenAI should be forced to develop such a "student mode" immediately and parents and educators need to be made aware of it to make sure students are using it, otherwise students are going to get much worse in tests, as they just ask it for answers to copy in assignments.

Re: Kids who use ChatGPT as a study assistant do worse on tests

#115

Post by hi_hi »

mtmickush wrote: post #98
hi_hi wrote: post #78

Story time. I always struggled with math as a kid. School to high school, then didn't touch it much until Uni. Teachers typically couldn't explain things in a way I "got it" in a school setting. I had some success with a private tutor to get me over the line in high school. Then at Uni I'm doing Computer Graphics, which included advanced (for me) math. I was panicked, and initially struggled until one of my good frie…

Glad to hear you were able to find a mechanism that clicked and stuck with it after that! The concept of learning styles for individuals though is a common myth https://onlineteaching.umich.edu/articles/the-myth-of-learni... https://www.youtube.com/watch?v=rhgwIhB58PA&t=2s There isn't any evidence that individuals learn best to a single style and generally approaching learning from multiple facets is the best way for…

Thankyou, I'll be interested to understand more about what the latest thinking around this is.

I've always assumed some people are better suited to learning one way, while other another way. I've never been good at absorbing information or understanding (which I'd differentiate from knowing, rightly or wrongly) through reading, while I know many others who can easily do this.

I'm hoping some of these resources touch on that.

Re: Kids who use ChatGPT as a study assistant do worse on tests

#116

Post by cageface »

eks391 wrote: post #102
cageface wrote: post #92

Earlier quoted context omitted.

What previous tools would you say replaced basic reasoning? I would not classify calculators, spreadsheets, spelling and grammar checkers etc as basic reasoning tools. I'm talking about bringing together several independent concepts or sets of facts and forming a coherent point of view or argument from them.

> What previous tools would you say replaced basic reasoning? No tool replaces reasoning itself. Not even an LLM. All (most) tools have the capacity to, if you use them for this purpose. > I would not classify calculators, spreadsheets, spelling and grammar checkers etc as basic reasoning tools. Why not? I have seen people use calculators to chug "6-2" before, and they were not children. In fact, all four of those ex…

> They create the opportunity to do great things, or be a crutch so you don't have to think

The very reason people are so excited about LLMs is that they believe that they are capable of a kind of thought that machines up to now have been incapable of. If you buy this then it follows that relying on them has a qualitatively different effect on our own abilities than previous tools.

Re: Kids who use ChatGPT as a study assistant do worse on tests

#117

Post by godelski »

alabhyajindal wrote: post #24

I recently used AI assistants for help with programming homework. My usual prompts include "help me think in the right direction", "is my thinking correct" etc. I also find myself copy pasting a question in chat to understand it better. I had the suspicion that this is not aiding in my learning process even though I am able to "solve" more problems. Nice to see this confirmed. Time to stop!

Key part of the study is that it gets the logical steps wrong. Worse, it's convincing, so be careful.

LLMs are great for finding key words in a domain you don't know which you can then use to search.

As a mostly self taught programmer the advice I'll give it to read docs and learn to read code. This is when my skills really increased. It's easy to get caught up in trying to just find the answer and doing these things definitely takes longer. But it has a multiplicative effect. You'll not only learn to be able to do it faster but you'll need to do it less often. So never pass up the opportunity to learn. There are times you need to rush but they're far less often than you think.

The other big advice I have is two strategies when coding. First write it quick and fast. Sloppy is okay, you're learning to solve the problem (move fast and break things). Then while you're cleaning up the mess you made, document. While you're doing that it'll stress the lessons into your memory and you'll almost always discovery things you missed. It is literally the rubber ducky method. Plus, you get the benefit of docs (others will read your code. And months or years later you'll come back, asking what idiot wrote this garbage to only find it's you. But also, that's a good sign, because you improved!). The second point is that the best code is flexible code. Remember that you suck at code (the secret is we all do) and you're going to have to come back, edit, and debug. So by writing flexible code you're making life easier for future you. It's really easy to forget these lessons because we think in the moment and our egos don't want us to admit we're bumbling idiots, but like in a video game, when it gets harder it means you're progressing. Don't misinterpret that signal.

Re: Kids who use ChatGPT as a study assistant do worse on tests

#119

Post by namaria »

llm_trw wrote: post #51
lll-o-lll wrote: post #5

When I was young, and learning math, my father always forbade me from looking at the answer in the back of the textbook. “You don’t work backwards from the answer!”, and I think this is right. In life, we rarely have the answer in front of us, we have to work that out from the things we know. It’s this struggling that builds a muscle you can then apply to any problem. ChatGPT, I suspect, is akin to looking up the ans…

In life there usually isn't an answer.

There's never an answer out there waiting for us. Answers are by definition created by people.

Re: Kids who use ChatGPT as a study assistant do worse on tests

#120

Post by ffsm8 »

wisty wrote: post #77
notatoad wrote: post #65

Earlier quoted context omitted.

>In life, we rarely have the answer in front of us, this sounds a lot like the "when you're an adult, you won't always have a calculator in your pocket" line we all heard in elementary school. and of course, we all know now how wrong that was. in life, chatGPT exists. and tools like it are only going to become more widespread. we are going to have the answers in front of us. knowing the things an LLM knows is not a u…

Not true. Current research is that experts are better with a ChatGTP, and beginners worse. An expert knows what to ask, and a beginner juat flails around. Tools like calculators, ChatGTP, Google, and chainsaws are often force magnifiers for experts, and a danger for beginners. If you have an idea of what you are trying to do at a deep level then they are great, but beginners don't, they usually just use them as a sho…

> Current research is that experts are better with a ChatGTP, and beginners worse.

Current research being your opinion?

Because literally every expert I personally know to be competent says that they're essentially garbage producers and don't use them beyond occasionally checking if they've improved. And no, they haven't.

The only way to get decent code from them is to be so specific that you'd be able to produce the same code in a fraction of the time you need to add all the context necessary for the LLM to figure it out.

It's like coding with a junior. Thay can produce great code too. It's just gonna take a lot of handholding if complexity skyrocks.

The only place llms got a foot hold is with generating stock art that's mostly soulless marketing material. This "industry" is going to crash so hard in the next 3 years

Post reply on HN