Absolutely horifying.
AI-generated videos to maximally drive a target brain region
201–210 of 258 posts
Re: AI-generated videos to maximally drive a target brain region
#202I’ll leave it to other comments to discuss the societal and moral implications of being able to do this (which, I agree, ick…). On a practical level: We train an encoding model, a “digital twin”, that predicts how each visual region responds to any video. Now we can ask: which video would make a chosen region light up the most? NEvo searches for that video automatically, using the twin’s prediction as its reward. I o…
You would like to buy some paperclips.
Re: AI-generated videos to maximally drive a target brain region
#203I’ll leave it to other comments to discuss the societal and moral implications of being able to do this (which, I agree, ick…). On a practical level: We train an encoding model, a “digital twin”, that predicts how each visual region responds to any video. Now we can ask: which video would make a chosen region light up the most? NEvo searches for that video automatically, using the twin’s prediction as its reward. I o…
A. Unless you think that brain science is immoral science, then I do not see any problem with this kind of research. As a neuroscientist, I strongly object to the insinuation as unfounded.
B. "Digital twins" e.g. [1] are a growing class of brain simulations that can successfully approximate brain activity patterns at large scale. I think these can be very useful, but we shouldn't think that they are at the level of actually simulating a brain. They are usually made of model neuronal approximative simulations (e.g. integrate and fire, balancing excitatory and inhibitatory neural populations within units), then using diffusion imaging to estimate white matter axonal wiring between those populations from the subject to increase the accuracy of the simulation. These are increasingly being used to, for example, model how a surgical intervention would effect seizure propagation prior to actual surgery. Here is a nice episode of Theoretical Neuroscience podcast [2] on the Virtual Brain [3], one of the available models for this kind of work.
C. In terms of validation. Only partly. From my quick read, this NEVO model optimized neural response only in the digital twin encoding model. While the digital twin model reportedly has solid predictive validity [4], which by the way was not the Virtual Brain model I mentioned in point B. Moreover, the outputs looked neurobiologically plausible, but at this point, there is no independent model or new fMRI showing the optimized stimuli actually drive the target regions. This was performed using previously collected fMRI data, and full validation of this model *IS* the obvious next step, but the money to collect such data does not come from nowhere: funding will be needed, and such a paper as this can help them get it.
D. A final point I'd make. We have long been able to create static stimuli that we can be fairly certain will activate above baseline certain brain regions, on average. Certain stimuli-region pairs ar emore homogenous between people, others e.g. the fusiform face area (FFA), are small enough that individual differences prevent a simple ROI approach, and identification depends on using face stimuli to identify at the individual level, but for the most part, it is reliably locatable. Brain activations are very coarse things. In fMRI, you are talking about ~3x3x3mm voxels (27mm^3) where the hemodynamic responses have a ton of spatial autocorrelation, or in EEG, where the surface spatial area of the reeptive fields are very large(~400 mm^2). These virtual twin models already do a decent job of modeling dynamics of the brain there parameters are tuned to *at this scale*..but this scale does not have a ton of information content. Automating this with video content is not that much a reach.
[1] https://spj.science.org/doi/10.34133/icomputing.0055 [2] https://theoreticalneuroscience.no/thn23/ [3] https://www.thevirtualbrain.org/tvb/zwei/ [4] https://www.biorxiv.org/content/10.1101/2025.07.22.664908v2....
Re: AI-generated videos to maximally drive a target brain region
#204I’ll leave it to other comments to discuss the societal and moral implications of being able to do this (which, I agree, ick…). On a practical level: We train an encoding model, a “digital twin”, that predicts how each visual region responds to any video. Now we can ask: which video would make a chosen region light up the most? NEvo searches for that video automatically, using the twin’s prediction as its reward. I o…
... and shouldn't this backfire hard? All the signals that are missed by the (f)MRI are never "mirrored" in the digital twin even though on screen it will look like it. Bam, Experimenter Bias in the machine and I don't know how to phrase it but ... does this method/experiment leave any wiggle room for Falsifiability? Even if the Hawthorne Effect does not apply to humans, it most certainly translates onto the brain an…
yeah. This is one of those things where they used previously collected data, built a model that has validity on modeling validity between simulation and actual brain (encoding model for digital twin), then built a new model using the encoding model that tries to reliably activate particular brain regions. The next obvious step is to validate this back to the original person, but they will need to collect new data, and so likely, will need to get research funding. One step at a time.
The digital twin types of models will become more and more useful, but I don't think we will be at the point where all research will be done entirely within these models.
They can be fun to play with btw: https://www.thevirtualbrain.org/tvb/zwei/ and they don't require a beast of a machine to run.
Re: AI-generated videos to maximally drive a target brain region
#205This is very similar to last week with that mind reading startup thing. Please read the paper before commenting. This is a tool to help researchers in figuring out what different parts of the brain are actually for with less experimenter bias contamination of “well we think maybe it’s about this so let’s show it video of x to see”. The essence runs on having someone sit in a scanner for a couple hours watching all so…
What is a "purpose"? Something people wish something would only be used for, right? How does it relate to, what influence does it have on what something will end up being used for?
Oh you are right. Let's just end neuroscience altogether. Also computer science, aerospace, and biology in general. In fact, let's go full Amish. Wait, no. Someone might use a buggy cart to run people over.
What exactly is your new fear here?
Re: AI-generated videos to maximally drive a target brain region
#206I’ll leave it to other comments to discuss the societal and moral implications of being able to do this (which, I agree, ick…). On a practical level: We train an encoding model, a “digital twin”, that predicts how each visual region responds to any video. Now we can ask: which video would make a chosen region light up the most? NEvo searches for that video automatically, using the twin’s prediction as its reward. I o…
Yeah, this smells very fishy to me. It's almost trivially easy to gather a small validation dataset in humans for the paper. At my institution, it's about $800 an hour to scan someone. You can probably get enough data to validate the model with a half hour scan. Surely the group has enough grant funding to pop a few healthy controls in the scanner.
I haven't looked in super close detail to the paper, but their methods section says that they fit a video model (V-JEPA2) to the fMRI dataset in a voxelwise ridge regression, meaning that the baked in assumption is that the visual response affects each voxel independently. Voxelwise models are very nice for making statistical inferences, but are less good for prediction and modelling tasks, because our brains certainly do not work as collections of independent regions.
BOLD is intensely messy data, and their design is far too simple IMO to reflect anything of reality.
Re: AI-generated videos to maximally drive a target brain region
#207This is the absolutely horrific next stage for social media platforms: - They're already well able to surface the most addictive short video for a specific user out of millions of real videos. - But these millions of real videos are just darts thrown into the space of "videos that could hook the user", in the end even the best-selected of them is not perfect. - Now, behold! AI allows to generate the perfect video to…
All glory to hypnotoad!
That would be ideal, but I think we're heading straight for Idiocracy's 'Ow! My Balls'.
Re: AI-generated videos to maximally drive a target brain region
#208An anecdote, but some reason to think that overworking part of your brain is a bad idea: I had an Aunt who had dementia. This is obviously a terrible outlook. But her and my Uncle seemed to be doing okay. My Uncle was an ultra-competent guy, highly stable, the person you could rely on; and had been his whole adult life. So it was shocking when he had a mental breakdown and became manic. There was probably something p…
A completely off-topic anecdote, then a completely unfounded fear based on a clear misunderstanding of this research. If companies ever start making us live in a MRI machine so they can reliably activate brain areas with certain stimuli, I think there is a bigger problem we are having.
Here is how I see it. This type of research helps us understand the brain, helps us do things like model potential surgery sites better (e.g. for seizure activity interventions). What it does not do is become the basis for mind reading.
Just so that you know that I am not totally unsympathetic to your apparent worldview, let me tell you how I think your concern might actually play out:
AI continues to get more powerful, and computer brain interfaces begin to move beyond EEG scalp electrodes, but begins to take the form or brain augmentation via integrating networked AI compute into implanted chips with electrode neural interfaces, where our brain's neurons and the neural interfaces learn to speak each other's language (i.e. integrate). I can actually see this happening within 20 years. At first, it would be our brain using this augmentation for greater intelligence, etc. However, remote manipulation of the interface could reverse control of brain activity, leading exactly to your fear.
But this research? This is light years from that,,,so much so that it is not even relevant to bring up unless you are just against all technology.
Re: AI-generated videos to maximally drive a target brain region
#209Earlier quoted context omitted.
I will state plainly that I do not think you are asking this maliciously, and that instead you are asking from curiosity. From that perspective, though, I am a bit flabbergasted. Are you truly curious as to how it might not be A Good Thing that those engaging in high-finance capitalistic money making programs would have access to forcefully engaging a human being's brain into activity, with or without their consent,…
> Are you truly curious as to how it might not be A Good Thing that those engaging in high-finance capitalistic money making programs would have access to forcefully engaging a human being's brain into activity, with or without their consent, to such an extent as to drive motivations and bias neuronal activation that leads to what they think is a choice? I guess I just don't think this is the topic at hand. What you'…
Its like we don't already show faces in fMRI experiments because we know that it will often activate the fusiform face area. Its just light years away from anything dangerous, that I am compelled, as a neuroscientist, to try to shake some sense into people. This is HN, but it feels like twitter on some topics.
Re: AI-generated videos to maximally drive a target brain region
#210This is the absolutely horrific next stage for social media platforms: - They're already well able to surface the most addictive short video for a specific user out of millions of real videos. - But these millions of real videos are just darts thrown into the space of "videos that could hook the user", in the end even the best-selected of them is not perfect. - Now, behold! AI allows to generate the perfect video to…
Social media is not even the worst threat vector, in my opinion. It’s Anthropic/OpenAI/Google, whom people are giving an unprecedented level of access to their innermost thoughts. We have not yet seen* the kind of large-scale, individually targeted psychological manipulation that cloud AI products can deliver. And I have no doubt the likes of Dario Amodei and Sam Altman will show us, if we give them enough time. * I…
> We have not yet seen* the kind of large-scale, individually targeted psychological manipulation that cloud AI products can deliver.
We did. Pre-WW II ..., and afterwards, the old, everlasting Cold War (vs the new one). And there are self-emerging/self-organizing buckets of people who all pledged their allegiance to ideas and plans, works forever in progress and aggregating jobs and expanding industries. People with value and virtue systems, radicalized or called to some need-to-know duty. And the personal level was always covered by applied and philosophical psychology, cultural discourses, however progressively or decadently lead by public figures while peoples' desires were and still are shaped by interactions, social AND parasocial.
Shouldn't we stop wondering whether they have enough time, now that several hundreds of thousands of hours of work already done and work yet to get done are neatly compressed into a few instants?
AI will always be "two" things:
- a search engine that misses less and less and is just waiting for you to increase your processing power - a tool to cope with your laziness or disability by deferring or skipping intervals of learning curves and/or biomechanics - an evergrowing compendium of abstractions and intents, formalized or not, by humans, for humans, off humans (even when it's AI)
but that's irrelevant ... because humans will always be only "two" things, as well.
So, ... the threat vector remains unchanged, however AI, accessibility and skill will amplify or change the landscape: casually, no, leisurely ignorant bystanders.