Kids who use ChatGPT as a study assistant do worse on tests

> What previous tools would you say replaced basic reasoning?

No tool replaces reasoning itself. Not even an LLM. All (most) tools have the capacity to, if you use them for this purpose.

> I would not classify calculators, spreadsheets, spelling and grammar checkers etc as basic reasoning tools.

Why not? I have seen people use calculators to chug "6-2" before, and they were not children. In fact, all four of those examples are perfect for my point. They create the opportunity to do great things, or be a crutch so you don't have to think. If you are convinsed that there is no user of grammarly that didn't just quickly type a draft and throw it in without proofreading, for the sake of not needing to use basic grammatical reasoning, then you have a very kind view of mankind.

> I'm talking about bringing together several independent concepts or sets of facts and forming a coherent point of view or argument from them.

This is a more narrow view than what I was arguing, so it's possible we will have to agree to disagree. I guess a quick rebuttle here is that people who don't exercise sufficient thought to form coherent conclusions from separate information were going to stay stupid anyway. And then yes, ChatGPT would be bad because it could in theory enable this behavior in its handouts.

After skimming through the article - as I understand ChatGPT was used as a tutor.

Many users of ChatGPT clearly know that it doesn't do math.

Now imagine your teacher is wrong at solving 40+% of problems it’s teaching you? Or that your measuring tape is wrong at nearly every second measurement?

Yeah it’s a tool - but you must nail your fundamentals right. Where I grew up - calculators were not allowed in elementary school - so every student must nail down basic arithmetic.

I think the part of overconfidence is actually important here. Looking at the paper Figure 2 shows the number of questions. For all sessions with GPT students averaged 2 messages per problem. Interestingly 20% isn't restating the problem or asking for the answer. Looks like 50-60% of tutor questions are asking for answer or restating problem and they ask more questions.

In my experience, usually where GPT and other LLMs get things wrong is in the steps. I see people are frequently over confident about the results and I've long thought this was a big part of it that the answer may be right but steps to get there are wrong. I see this a lot with river crossing problems. People will often show a prompt that solves it (half the time there's information leakage) and a good portion of those will have the logic errors. But people just look at the answer (btw, this is the reason I claim LLMs don't reason. It's not about getting the answer wrong, it's about the logical steps. That's the real evaluation!)

Personally I think this is good evidence of over fitting (memorization).

I also think this is why you should be careful when using it to code. It's the details that matter a lot. Since LLMs are aggregators I'd like to remind everyone that the average coder and average code is terrible. Hell, even good programmers often suck. Code is hard! Whenever I try using LLMs to code I find that I write lines faster but I end up spending more time debugging and prompt engineering than if I just read the docs and did it myself (this also has the added benefit of the struggle making me remember more). So I'll use it to write things I don't really care about but otherwise I'm not getting how people are finding it so helpful (yes, I read my comment). But I suspect many people are similar to the students in the study.

[dead]

Disagree that it's the equivalent of SO. Unlike SO, I can ask it why it is doing something and as a result I can better understand what it writes, I can create little test cases, and make sure it's not going to do anything destructive or incorrect.

Given the fact that I only ever write bash scripts when I need to "make computer do thing now now now now now" the likelihood that I would ever take the time to deeply learn the ins and outs of bash are slim to none.

I can ask it to explain certain constructs and give me worked examples very quickly to improve my understanding.

Even though I can't say I could reproduce all the various things that it has written for me, I know things that bash can do that I was never aware of before, and I can recognise lots of things in the syntax now that would previously have looked like gibberish.