As someone with aphantasia I'd love to see this used to work out what it is that our brains aren't doing.
[deleted]
Real-Time Brain-to-Image Reconstructions
11–20 of 27 posts
Re: Real-Time Brain-to-Image Reconstructions
#12Re: Real-Time Brain-to-Image Reconstructions
#13For the love of God stop using Discord to organize your community and hold all your discussions, especially scientific endeavors! you are bound to lose everything and have immense difficulty searching for all the past information in the future
I feel this struggle although wonder what you recommend instead (besides slack ;)?
Re: Real-Time Brain-to-Image Reconstructions
#14Not much to see there. I don't have much faith that the stated goal is realizable, I don't think fMRI data holds enough information for faithful reconstruction. What it will end up being, like all projects like it before, is a massive network that contains: - An implicit multi-class classifier that returns 1 out of n previously seen images - Gigantic model on top to hallucinate a reconstruction close to that previous…
I don’t know. Now that you explain how it works, it’s starting to worry me. Imagine being hooked up to an ML model designed to classify whether you’re thinking about porn or not, and emits a beep whenever it detects it. The implications of that are astonishing, if it’s accurate. It hinges on accuracy. If it can detect a topic, the image reconstruction is almost moot, because you can start interrogating people about w…
> Imagine being hooked up to an ML model designed to classify whether you’re thinking about porn or not
Or whether you're thinking thoughts the government doesn't like.
Re: Real-Time Brain-to-Image Reconstructions
#15For the love of God stop using Discord to organize your community and hold all your discussions, especially scientific endeavors! you are bound to lose everything and have immense difficulty searching for all the past information in the future
I feel this struggle although wonder what you recommend instead (besides slack ;)?
For off the record discussions you can host a Matrix server. For recorded meeting you can use Jitsi. For recorded discussions and knowledge you can set up a wiki or a forum
Re: Real-Time Brain-to-Image Reconstructions
#16fMRI will never have the resolution necessary for something like this. But Neuralink or other precision implants might, if they can put enough probes in enough of the right places.
Re: Real-Time Brain-to-Image Reconstructions
#17Not much to see there. I don't have much faith that the stated goal is realizable, I don't think fMRI data holds enough information for faithful reconstruction. What it will end up being, like all projects like it before, is a massive network that contains: - An implicit multi-class classifier that returns 1 out of n previously seen images - Gigantic model on top to hallucinate a reconstruction close to that previous…
I generally agree with the assessment that the limiting factor in quality is the signal from the MRI. Just one slight correction to the description of previous projects. We first built models that predict the voxel responses from images using ML. Then recorded the fMRI response on a new image. Then we ran many many images (not just those previously seen that have an fMRI response) through the voxel models and picked the top N images with predicted responses close to the actual response. Basically we used the big list of images as an approximate "natural image prior". Then we'd often show the average of that top N images so you get one picture (which destroys interesting multi-model stuff, but we didn't have good stable diffusion models of images back then that could do a gradient search on or something).
If you wanna see what the reconstructions look like without fancy image priors (to get a better idea of how little signal there is) check out this figure from one of the early papers: https://pubmed.ncbi.nlm.nih.gov/19778517/#&gid=article-figur...
Re: Real-Time Brain-to-Image Reconstructions
#18As someone with aphantasia I'd love to see this used to work out what it is that our brains aren't doing.
I can't even imagine (no pun intended) what would be like having aphantasia.
It would be weird to me not being able to draw from memory a map of my own home.
Or not being able to answer a question like "if you enter your home, walk straight ahead until you reach a wall, and then turn 90° counterclockwise, what would you be looking at?".
Unless I'm wildly misunderstanding what "having a mental image" means.
Re: Real-Time Brain-to-Image Reconstructions
#19This technology one can imagine will be used in the future in books about a digital dictatorship , where the author has computers monitor the workers brains in society who are operating airport scanners and other security/important control jobs .... who might be tempted to not follow the dictatorships increasingly crazy rules or for example allow someone/something through in a critical situation that could lead to an attack by the societies people in the name of freedom.
"The culmination of all the above projects. We will consolidate our findings to ultimately reconstruct images from brain activations in real-time (i.e., reconstruct a seen image in a few seconds following perception, while the patient is still being scanned)."
Re: Real-Time Brain-to-Image Reconstructions
#20This technology one can imagine will be used in the future in books about a digital dictatorship , where the author has computers monitor the workers brains in society who are operating airport scanners and other security/important control jobs .... who might be tempted to not follow the dictatorships increasingly crazy rules or for example allow someone/something through in a critical situation that could lead to an…
"I was told that if I wanted to, I could vote with my feet and choose a job that didn't use B2I. The only problem was, they didn't exist anymore. What good is a choice when only one option exists?"