51–60 of 69 posts
Aside from helping those with disabilities, what is stopping authorities using this as a lie detector? I assume the tech isn’t quite there yet.
Not yet. > Models were trained separately for every participant and are not generalizable across people.
Awesome, predicting words from fMRI has been around for a while and visual cortex can be mapped well. That said, and coming from a background in neuroimaging 20 years ago, what’s the applicability? MRI hasn’t gotten that much more cost effective for more widespread uses. Magnets are expensive.
We think it could be useful for clinical research and maybe even diagnostics. For example, you could imagine a person with depression(or other neurological disorders) may have a different perception of the same image than a healthy person. Now with the much higher fidelity that both more powerful MRI machines and better generative AI tools can provide, this may now be a very promising direction for future research.
Wasn't there something similar a few months ago on HN and where the top comment talked about how it's not as impressive as it sounds [0]? The main issue is that this type of methodology is pulling from a pool of images, not literally reconstructing what image was seen in the brain directly. > I immediately found the results suspect, and think I have found what is actually going on. The dataset it was trained on was 2…
Our model generates CLIP image embeddings from fMRI signals and those image embeddings can be used for retrieval (using cosine similarity for example) or passed into a pretrained diffusion model that takes in CLIP image embeddings and generates an image (it's a bit more complicated than that but that's the gist, read the blog post for more info). So we are doing both reconstruction and retrieval. The reconstruction a…
Wasn't there something similar a few months ago on HN and where the top comment talked about how it's not as impressive as it sounds [0]? The main issue is that this type of methodology is pulling from a pool of images, not literally reconstructing what image was seen in the brain directly. > I immediately found the results suspect, and think I have found what is actually going on. The dataset it was trained on was 2…
Our model generates CLIP image embeddings from fMRI signals and those image embeddings can be used for retrieval (using cosine similarity for example) or passed into a pretrained diffusion model that takes in CLIP image embeddings and generates an image (it's a bit more complicated than that but that's the gist, read the blog post for more info). So we are doing both reconstruction and retrieval. The reconstruction a…
For context, early vision is easier to map than you might expect. Here's a radiograph of the primary visual cortex created in 1982 by projecting a pattern onto a macaque's retina: https://web.archive.org/web/20100814085656im_/http://hubel.m... An injection of radioactive sugar lets you see where the neurons were firing away and metabolizing the sugar. ( https://pubmed.ncbi.nlm.nih.gov/7134981/ )
But can brain activity be mapped anywhere like this with fmri? I doubt it. But yes — it is cool that the brain keeps spatial proportions of reality in the brain map! Very unlike latent space.
Fig4 shows the letter M on the cortical surface, where the stimulus accounted for the effects of foveal magnification (foveal vision gets more cortical space). Keep in mind that we now, in theory, have stronger magnets, better head coils (the part that picks up the image information), and better sequences (the software that manipulates the magnets to produce the images) so we could do even better than that these days.
Earlier quoted context omitted.
Our model generates CLIP image embeddings from fMRI signals and those image embeddings can be used for retrieval (using cosine similarity for example) or passed into a pretrained diffusion model that takes in CLIP image embeddings and generates an image (it's a bit more complicated than that but that's the gist, read the blog post for more info). So we are doing both reconstruction and retrieval. The reconstruction a…
Why are you building this, and what kind of ethical considerations have you taken, if any?
There's a inevitability about most scientific discoveries (there are notable exceptions but they are few) and unless we're talking about something with capital outlay in the trillions of dollars then it's going to happen whether we like it or not - short of a global totalitarian state capable of deep scrutiny of all research.
Earlier quoted context omitted.
Why are you building this, and what kind of ethical considerations have you taken, if any?
I'm curious what answers you would find acceptable? I'm not being snarky - I genuinely struggle with this line of thinking. People seem to find "if I don't then someone else will" to be an unacceptable answer but it seems to me to be fairly central. There's a inevitability about most scientific discoveries (there are notable exceptions but they are few) and unless we're talking about something with capital outlay in…
Because you can use this as a cop out for truly heinous work. I.e. gain of function research, autonomous weapons, chemical weapons, etc. It's not a coherent world view for someone that actually cares about doing good.
This is SO COOL. I'd guess (I did analysis for an fMRI lab for a year so I'm not a pro but not totally talking out of my orifice) that detecting images like this is among the easier things you could do (it probably wouldn't be so easy to do things like "guess the words I'm thinking of") and I suspect other sensory stuff might be harder but I have little knowledge there. One of the biggest issues with any attempt to e…
Yeah using data from a 7T MRI giving higher spatial resolution definitely helps! The fMRI dataset includes signal from the whole brain but we only use the data from the visual cortex for this study.
Earlier quoted context omitted.
I'm curious what answers you would find acceptable? I'm not being snarky - I genuinely struggle with this line of thinking. People seem to find "if I don't then someone else will" to be an unacceptable answer but it seems to me to be fairly central. There's a inevitability about most scientific discoveries (there are notable exceptions but they are few) and unless we're talking about something with capital outlay in…
>People seem to find "if I don't then someone else will" to be an unacceptable answer but it seems to me to be fairly central. Because you can use this as a cop out for truly heinous work. I.e. gain of function research, autonomous weapons, chemical weapons, etc. It's not a coherent world view for someone that actually cares about doing good.
When research has an obvious and immediate negative outcome that's a cost. The difficulty/expense of the research is also a cost.
The "benefit" would be the incentive to know the outcome. This may be profit, military advantage, academic kudos etc.
Maybe the problem with the type of research being discussed here is that there isn't neccesarily any agreement that the outcome is negative. For many people, I suspect this will remove a lot of the weight on the "cost" side of things.
I'm not making a specific point here - I'm actually trying to work this out in my head as I write.