Live data from Hacker News

Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

media.mit.edu

171–180 of 529 posts

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#171

idk, if anything I’m thinking more. The idea that I might be able to build everything I’ve ever planned out. At least the way I’m using them, it’s like the perfect assistive device for my flavor of ADHD — I get an interactive notebook I can talk through crazy stuff with. No panacea for sure, but I’m so much higher functioning it’s surreal. I’m not even using em in the volume many folks claim, more like pair programmi…

[deleted]

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#172
post #82
post #36

Earlier quoted context omitted.

> Similar to the mass psychosis we were hearing about during COVID Can you be more specific and/or provide some references? The "demonstrating curiosity about controversial topics" part is sounding like vaccine skepticism, though I don't recall ever hearing that being referred to as any kind of "psychosis".

Noting that it is straw man to connect my argument with vaccine skepticism. The mass psychosis was that early on in the COVID response, we were hearing so much early advice from people that were ahead of CDC/FDA, things like: - Masks work (CDC/FDA discouraged, then flip-flopped and took credit for these things) despite it originating from Scott Alexander and skeptic communities like his, I also heard it from Tim Ferr…

>Masks work (CDC/FDA discouraged, then flip-flopped

Discouraging them early on was meant to avoid supply runs on quality masks. I agree it was a misstep on their part to promote the falsehood that masks do not prevent the wearer from being infected, and they never sufficiently walked this back, only perpetuating further myths like masks only protect others and not the wearer.

>Asking about "Hey if I got COVID before, that immunity is as robust if not more than vaccine, what evidence supports I need the vaccine?"

I also agree that over-reliance and perhaps overselling of vaccine effectiveness was a misstep, largely designed to get societal buy-in for ignoring COVID and "getting back to normal" as quickly as possible. The point that makes suspect those who were in favor of things like vitamins and exercise and so adamantly against measures like vaccines is that they did not go on to support other mitigations to promote health, like mask mandates and improvements in indoor air filtration and ventilation, which would have been more effective at reducing disease and promoting health. On the contrary, such activists were only interested in removing all measures and promoting increased disease.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#173
post #45

I think a lot more people, especially at the higher end of the pay scale, are in some kind of AI psychosis. I have heard people at work talk about how they are using chatGPT to quick health advice, some are asking it for gym advice and others are just saying they just dump entire research reports into it and get the summary.

What does using a chat agent have to do with psychosis? I assume this was also the case when people googled their health results, googled their gym advice and googled for research paper summaries? As long as you're vetting your results just like you would any other piece of information on the internet then it's an evolution of data retrieval.

https://en.wikipedia.org/wiki/Chatbot_psychosis

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#174

This reminds me of the recurring pattern with every new medium: Socrates worried writing would destroy memory, Gutenberg's critics feared for contemplation, novels were "brain softening," TV was the "idiot box." That said, I'm not sure "they've always been wrong before" proves they're wrong now. Where I'm skeptical of this study: - 54 participants, only 18 in the critical 4th session - 4 months is barely enough time…

> "they've always been wrong before"

In my opinion, they've almost always been right.

In the past two decades, we've seen the less-tech-savvy middle managers who devalued anything done on computer. They seemed to believe that doing graphic design or digital painting was just pressing a few buttons on the keyboard and the computer would do the job for you. These people were constantly mocked among online communities.

In programmers' world, you have seen people who said "how hard it could be? It's just adding a new button/changing the font/whatever..."

And strangely, in the end those tech muggles were the insightful ones.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#175
post #98

This seems to confirm my feeling when using AI too much. It's easy to get started, but I can feel my brain engaging less with the problem than I'm used to. It can form a barrier to real understanding, and keeps me out of my flow. I recently worked on something very complex I don't think I would have been able to tackle as quickly without AI; a hierarchical graph layout algorithm based on the Sugiyama framework, using…

> a hierarchical graph layout algorithm based on the Sugiyama framework, using Brandes-Köpf for node positioning. I am sorry for being direct but you could have just kept it to the first part of that sentence. Everything after that just sounds like pretentious name dropping and adds nothing to your point. But I fully agree, for complex problems that require insight, LLMs can waste your time with their sycophancy.

This is a technical forum, isn't pretentious name dropping kind of what we do?

Seriously though, I appreciated it because my curiosity got the better of me and I went down a quick rabbit hole in Sugiyama, comparative graph algorithms, and learning about the node positioning as a particular dimension of graph theory. Sure nothing ground breaking, but it added a shallow amount to my broad knowledge base of theory that continues to prove useful in our business (often knowing what you don't know is the best initiative for learning). So yeah man, lets keep name dropping pretentious technical details because thats half the reason I surf this site.

And yes, I did use ChatGPT to familiarize myself with these concepts briefly.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#176
post #102

Earlier quoted context omitted.

This mirrors my experience exactly. We have to learn how to tame the beast.

I think we all just need to avoid the trap of using AI to circumvent understanding. I think that’s where most problems with AI lie. If I understand a problem and AI is just helping me write or refactor code, that’s all good. If I don’t understand a problem and I’m using AI to help me investigate the codebase or help me debug, that’s okay too. But if I ever just let the AI do its thing without understanding what it’s…

I'd say the new problem is knowing when understanding is important and where it's okay to delegate.

It's similar to other abstractions in this way, but on a larger scale due to LLM having so many potential applications. And of course, due to the non-determinism.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#177
post #98

This seems to confirm my feeling when using AI too much. It's easy to get started, but I can feel my brain engaging less with the problem than I'm used to. It can form a barrier to real understanding, and keeps me out of my flow. I recently worked on something very complex I don't think I would have been able to tackle as quickly without AI; a hierarchical graph layout algorithm based on the Sugiyama framework, using…

That's a nice anecdote, and I agree with the sentiment - skill development comes from practice. It's tempting to see using AI as free lunch, but it comes with a cost in the form of skill atrophy. I reckon this is even the case when using it as an interactive encyclopedia, where you may lose some skill in searching and aggregating information, but for many people the overall trade off in terms of time and energy savin…

If the computer was the bicycle for the mind, then perhaps AI is the electric scooter for the mind? Gets you there, but doesn't necessarily help build the best healthy habits.

Trade offs around "room to do more of other things" are an interesting and recurring theme of these conversations. Like two opposites of a spectrum. On one end the ideal process oriented artisan taking the long way to mastery, on the other end the trailblazer moving fast and discovering entirely new things.

Comparing to the encyclopedia example: I'm already seeing my own skillset in researching online has atrophied and become less relevant. Both because the searching isn't as helpful and because my muscle memory for reaching for the chat window is shifting.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#178
ChatGPT got me over my imposter syndrome.

Back when it came out, it was all the rage at my company and we were all trying it for different things. After a while, I realized, if people were willing to accept the bullshit that LLMs put out, then I had been worrying about nothing all along.

That, plus getting an LLM to write anything with meaning takes putting the meaning in the prompt, pushed me to finally stop agonizing over emails and just write the damn things as simply and concisely as possible. I don't need a bullshit engine inflating my own words to say what I already know, just to have someone on the other end use the same bullshit engine to remove all that extra fluff to summarize. I can just write the point straight away and send it immediately.

You can literally just say anything in an email and nobody is going to say it's right or wrong, because they themselves don't know. Hell, they probably aren't even reading it. Most of the time I'm replying just to let someone know I read their email so they don't have to come to my office later and ask me if I read the email.

Every time someone says the latest release is a "game changer", I check back out of morbid curiosity. Still don't see what games have changed.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#179
post #167
post #98

This seems to confirm my feeling when using AI too much. It's easy to get started, but I can feel my brain engaging less with the problem than I'm used to. It can form a barrier to real understanding, and keeps me out of my flow. I recently worked on something very complex I don't think I would have been able to tackle as quickly without AI; a hierarchical graph layout algorithm based on the Sugiyama framework, using…

My "actual job" isn't to write code, but to solve problems. Writing code has just typically been how I've needed to solve those problems. That has increasingly shifted to "just" reviewing code and focusing on the architecture and domain models. I get to spend more time on my actual job.

This feels like it conflates problem solving with the production of artifacts. It seems highly possible to me that the explosion of ai generated code is ultimately creating more problems than it is solving and that the friction of manual coding may ultimately prove to be a great virtue.

Re: Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant

#180

I encourage folks to listen to brilliant psychologist for software teams Cat Hicks [1] and her wife, teaching neuroscientist Ashley Juavinett [2] on their excellent podcast, Change, Technically discussing the myriad problems with this study: https://www.buzzsprout.com/2396236/episodes/17378968 1: https://www.catharsisinsight.com 2: https://ashleyjuavinett.com

I'm not a fan of "TL;DR" but I think 52 minutes would qualify. I jumped to a random point of the transcript and found just platitudes, which didn't quite hook me into listening to all of it. How about some more info on what their main conclusions are?

They view the framing of the MIT paper not just as bad science, but as a dangerous social tool that uses brain data to "consign people" to being less worthy or "stupid" for using cognitive aids. It flags the paper's alarmist findings as "pseudoscience" designed to provoke fear rather than provide rigorous insight. They highlight several "red flags" in the study's design: lack of a coherent scientific framework, methodological errors like typos, and reliance on invented, undefined terms such as "cognitive debt". They challenge the interpretation of EEG results, explaining that while the paper frames a 55% reduction in connectivity as evidence that a user's "brain sucks," such data could instead indicate increased neural efficiency, an alternative explanation the authors ignore. (EEG measures broad, noisy signals from outside the skull and is better understood as a rough index of brain state than as a precise window into specific thoughts or “intelligence.”)

The hosts condemn the study’s "bafflingly weak" logic and ableist rhetoric, and advise skepticism toward "science communicators" who might profit from selling hardware or supplements related to their findings: one of the paper's lead authors, Nataliya Kosmyna, is associated with the MIT Media Lab and the development of AttentivU, a pair of glasses designed to monitor brain activity and engagement. By framing LLM use as creating a "cognitive debt," the researchers create a market for their own solution: hardware that monitors and alerts the user when they are "under-engaged". The AttentivU system can provide haptic or audio feedback when attention drops, essentially acting as the "scaffold" for the very cognitive deficits the paper warns against. The research is part of the "Fluid Interfaces" group at MIT, which frequently develops Brain-Computer Interface (BCI) systems like "Brain Switch" and "AVP-EEG". This context supports the hosts' suspicion that the paper’s "cognitive debt" theory may be designed to justify a need for these monitoring tools.

Post reply on HN