Live data from Hacker News

The Impact of Generative AI on Critical Thinking [pdf]

microsoft.com

41–50 of 109 posts

Re: The Impact of Generative AI on Critical Thinking [pdf]

#43
post #2

Study (PDF): https://www.microsoft.com/en-us/research/uploads/prod/2025/0... I did not link to it directly because the PDFs title - "The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers" - was way too long for the HN title length limit. Abstract (from the PDF): > The rise of Generative AI (GenAI) in knowledge workflows…

> Analysing 936 real-world GenAI tool use examples our participants shared, we find that knowledge workers engage in critical thinking primarily to ensure the quality of their work, e.g. by verifying outputs against external sources.

The researchers appear to be saying that a lot of people perceive LLM outputs to be superior to whatever they could generate themselves, so users accept the LLM's logical premises uncritically and move on. Sort of a "drive by" use of chatbots. The researchers are not wrong, but intuitively I don't think this is a fair critique of most LLM tasks, or even most users.

One of the most compelling LLM use cases is individualized learning or tutoring, and LLM's can support a much deeper dive into arcane scientific or cutting-edge topics than traditional methods. I don't see anything here that suggests the researchers balanced these scenarios against one another.

Re: The Impact of Generative AI on Critical Thinking [pdf]

#44
post #34

Any chance the link can be changed to the original paper ( https://www.microsoft.com/en-us/research/uploads/prod/2025/0... ) and the title changed to "The Impact of Generative AI on Critical Thinking"?

This would be even better:

https://www.microsoft.com/en-us/research/publication/the-imp...

Re: The Impact of Generative AI on Critical Thinking [pdf]

#45

I think this study does not say what most people are taking it to say. > our research questions - when and how knowledge workers perceive the enaction of critical thinking when using GenAI (RQ1), and when and why do knowledge workers perceive increased/decreased effort for critical thinking due to GenAI (RQ2) This is about the application of critical thinking to AI outputs and in AI workflows. Are people cognitively…

The study is basically just... a bunch of self-reports on how people feel about AI usage in their daily job. That's it. The "critical thinking" part is completely self-reported.

It's not "how good a person is at critical thinking." It's "how much a person feels they engages in critical thinking." Keyword: feels.

It's like all the nutrition studies where they ask people to self-report what they eat.

Re: The Impact of Generative AI on Critical Thinking [pdf]

#46
post #23

People are worried AI is making us dumber. You hear it all the time. GPS wrecked our sense of direction. Spellcheck killed spelling. Now it’s AI’s turn to supposedly rot our brains. It’s the same old story. New tool comes along, people freak out about what we’re “losing.” But they’re missing the point. It’s never about losing skills, it’s about shifting them. And usually, the shift is upwards. Take GPS. Yeah, okay, m…

This assumes you always have reliable GPS: - areas that lack map details and obstructions that preclude straight line paths, e.g. deep forest - wartime GPS blocking - device failure incl battery - errors in mapping data - maps that contain other information not found in your GPS-enabled map

So then carry a map or ask for directions when these situations arise? What does this have to do with the positive trade-offs when using GPS?

Re: The Impact of Generative AI on Critical Thinking [pdf]

#49
post #35

Interesting, in my experience LLMs hallucinate so much on stuffs I know about that I instinctively challenge most of their assumptions and outputs, and found out that this kind of dialectic exchange bring the most out of the "relationship" so to speak, co creating something greater than the isolation of us two. Relevant 2018 essay by Nicky Case «How To Become A Centaur»: https://jods.mitpress.mit.edu/pub/issue3-case/…

I haven't used LLMs a lot and have just experimented with them in terms of coding.

But about a year ago, I had a job to clean up a bunch of, let's call them, reference architectures. I mostly didn't mess with the actual architecture part or went directly to the original authors.

But there wasn't a lot of context setting and background for a lot of them. None of it was rocket science; I could have written myself. But the LLM I used, Bard at the time, gave me a pretty good v 0.9 for some introductory paragraphs. Nothing revelatory but probably saved me an hour or two per architecture even including my time to fixup the results. Generally, nothing was absolutely wrong but some I felt was less relevant and other stuff I felt was missing.

Re: The Impact of Generative AI on Critical Thinking [pdf]

#50

I think this study does not say what most people are taking it to say. > our research questions - when and how knowledge workers perceive the enaction of critical thinking when using GenAI (RQ1), and when and why do knowledge workers perceive increased/decreased effort for critical thinking due to GenAI (RQ2) This is about the application of critical thinking to AI outputs and in AI workflows. Are people cognitively…

It really has been a sight watching the loudest anti-AI people flog this around, then turn to rage when you clarify the range of the actual conclusion.

Generative AI got a lot more useful when I started seeing it abstractly. I put data in, it finds correlations in its neural network, and produces different data. I have to be intentional in interpreting that data to figure out what to do with it.

Once I started thinking in terms of "this word will kick up a hornet's nest in that branch of the network and make the output useless. Let's find another word that gets closer to what I'm aiming for," the outputs got so much better. And getting to the good outputs took fewer iterations.

Post reply on HN