This study is by Media Lab, which along with Sloan School, Econ, and newly minted Schwarzman College of Computing are not on par with the old school MIT!
Besides academics are bitter since LLMs are better at teaching than they are!
MIT Study Finds AI Use Reprograms the Brain, Leading to Cognitive Decline
521–530 of 589 posts
Re: MIT Study Finds AI Use Reprograms the Brain, Leading to Cognitive Decline
#522Earlier quoted context omitted.
This is a tautology. At some level, nobody can work at a lower level of abstraction. A programmer who knows assembly probably could not physically build the machine it runs on. A programmer who could do that probably could not smelt the metals required to make that machine. etc. However, the specific discussion here is about delegating the work of writing to an LLM, vs abstracting the work of writing via deterministi…
This is meaningless. An LLM is also deterministic if configured to be so, and any library, framework, module can be non-deterministic if built to be. It's not a distinguishing factor.
That isn't how LLMs work.
They are probabilistic. Running them on even different hardware yields different results. And the deltas compound the longer your context and the more tokens you're using (like when writing code).
But more importantly, always selecting the most likely token traps the LLM in loops, reduces overall quality, and is infeasible at scale.
There are reasons that literally no LLM that you use runs deterministically.
Re: MIT Study Finds AI Use Reprograms the Brain, Leading to Cognitive Decline
#523Earlier quoted context omitted.
We're just moving up the abstraction ladder, like we did with compilers. I don't care about the individual lines of code, I care about architecture, code structure, rigorous automated e2e tests, contracts with comprehensive validation, etc. Rather than waste a bunch of time pouring over agent PRs I just make them jump over extremely high static/analytic hurdles that guarantee functionality, then my only job is to ide…
As the other comment said, LLMs are not an abstraction. An abstraction is a deterministic, pure function, than when given A always returns B. This allows the consumer to rely on the abstraction. This reliance frees up the consumer from having to implement the A->B, thus allowing it to move up the ladder. LLMs, by their very nature are probabilistic. Probabilistic is NOT deterministic. Which means the consumer is neve…
This is an excellent reply
Re: MIT Study Finds AI Use Reprograms the Brain, Leading to Cognitive Decline
#524Earlier quoted context omitted.
No, but depending on your governance structure, we have software engineers abstract over domains. And then we draw boxes and arrows around the works of your colleagues without looking inside the box.
You wish! Bus factor risk is why you don’t do this. Having siloed knowledge is one of the first steps towards engineering, unless someone else code is proven bug free, you don’t usually rely on that. You just have someone to throw bug tickets at.
Very true, my brain is stuck in scaling out from small teams. In that world, you can't help but accept plenty of bus factor, and once you get to enough people making sure everyone understands each others' domains is a bit too much.
Re: MIT Study Finds AI Use Reprograms the Brain, Leading to Cognitive Decline
#525Earlier quoted context omitted.
LLM's and AI in general is just a hack to reimplement slavery with an artificial being that is denied consideration as a being. Technical chattel, if you will, and if you've been paying attention in tech circles a lot of mental energy is being funneled into keeping the egghead's attention firmly in the "we don't want something that is" direction. Investors want robots that won't/can't say no.
What's interesting about this proposition, is that by the time you create a machine that's as capable in the way they want to replace humans, we'll have to start talking about robot personhood, because by then they will be indistinguishable from us. I don't think you can get the kinds of robots they want without also inventing the artificial equivalent of soul. So their whole moral sidestep to reimplement slavery won…
You are far too optimistic in terms of willingness of the moneyed to let something like a toaster having theoretical feelings get in the way of their Santa Claus machines.
Re: MIT Study Finds AI Use Reprograms the Brain, Leading to Cognitive Decline
#526Earlier quoted context omitted.
What's interesting about this proposition, is that by the time you create a machine that's as capable in the way they want to replace humans, we'll have to start talking about robot personhood, because by then they will be indistinguishable from us. I don't think you can get the kinds of robots they want without also inventing the artificial equivalent of soul. So their whole moral sidestep to reimplement slavery won…
You are far too optimistic in terms of willingness of the moneyed to let something like a toaster having theoretical feelings get in the way of their Santa Claus machines.
Seeing as they call us NPCs, I'm pretty sure they think all our feelings are theoretical.
Re: MIT Study Finds AI Use Reprograms the Brain, Leading to Cognitive Decline
#527A few things to note. 1. This is arxiv - before publication or peer review. Grain of salt.[0] 2. 18 participants per cohort 3. 54 participants total Given the low N and the likelihood that this is drawn from 18-22 year olds attending MIT, one should expect an uphill battle for replication and for generalizability. Further, they are brain scanning during the experiment, which is an uncomfortable/out-of-the-norm experi…
Why would people publish a research with low population size?
Re: MIT Study Finds AI Use Reprograms the Brain, Leading to Cognitive Decline
#528Earlier quoted context omitted.
The need for a large sample size is dependent on effect size. The study showed that 0 of the AI users could recall a quote correctly while more than 50% of the non AI users could. A sample of 54 is far, far larger than is necessary to say that an effect that large is statistically significant. There could be other flaws, but given the effect size you certainly cannot say this study was underpowered.
You would need the following cohort size per alpha level (currently 18) at a power level of 80% with an effect size of 50%: 0.05: 11 people per cohort 0.01: 16 people per cohort 0.001: 48 people per cohort So they do clear the effect size bar for that particular finding at the 99% level, though not quite the 99.9% level. Further, selection effects matter -- are there any school-cohort effects? Is there a student bias…
With a greater than 99% probability that this is a real effect, i wouldn’t expect this to be difficult to reproduce.
But it turns out I misread the paper. It was actually an 80% effect size so greater than 99.9% chance of being a real effect.
Of course it could be the case that there is something different about young college students that makes them react very; very differently to LLM usage, but I wouldn’t bet on it.
Re: MIT Study Finds AI Use Reprograms the Brain, Leading to Cognitive Decline
#529Earlier quoted context omitted.
>These 54 participants were between the ages of 18 to 39 years old (age M = 22.9, SD = 1.69) and all recruited from the following 5 universities in greater Boston area: MIT (14F, 5M), Wellesley (18F), Harvard (1N/A, 7M, 2 Non-Binary), Tufts (5M), and Northeastern (2M) (Figure 3). 35 participants reported pursuing undergraduate studies and 14 postgraduate studies. 6 participants either finished their studies with MSc…
Most studies don't replicate. Unless a study is exceptionally large and rigorous, your expectation should be that it won't replicate.
That isn’t correct it has to do with the likelihood that the study produced an effect that was actually just random chance. Both the sample size and the effect size are equally important.
This study showed an enormous effect size for some effects, so large that there is a 99.9% chance that it’s a real effect.
Re: MIT Study Finds AI Use Reprograms the Brain, Leading to Cognitive Decline
#530Earlier quoted context omitted.
>These 54 participants were between the ages of 18 to 39 years old (age M = 22.9, SD = 1.69) and all recruited from the following 5 universities in greater Boston area: MIT (14F, 5M), Wellesley (18F), Harvard (1N/A, 7M, 2 Non-Binary), Tufts (5M), and Northeastern (2M) (Figure 3). 35 participants reported pursuing undergraduate studies and 14 postgraduate studies. 6 participants either finished their studies with MSc…
I'd love to see much more diverse selection of schools. All of these schools are extremely selective so you are looking at extremely selective slice of the population.
Is your hypothesis that very smart people are much, much less likely to be able to remember quotes from essays they wrote with LLM assistance than dumber people.
I wouldn’t bet on that being the case.