I don't see anything "high-res" about the reconstructed images.
High-res image reconstruction with latent diffusion models from human brain
71–80 of 170 posts
Re: High-res image reconstruction with latent diffusion models from human brain
#72Earlier quoted context omitted.
There was a guy at MIT about ten years ago (edit: 2018! Woah) who made a headset that would read electrical impulses from your face. Apparently when people think in words, the same nerves fire as when they speak, just at a lower activation level. Using those signals it is possible to reconstruct the words being thought. I'm surprised it didn't seem to go anywhere. Edit: found it https://youtu.be/RuUSc53Xpeg
> I'm surprised it didn't seem to go anywhere. At least not publicly.
There have been some corporate research groups that have tried to take this approach further, and they all have more or less failed as far as I know.
Re: High-res image reconstruction with latent diffusion models from human brain
#73Earlier quoted context omitted.
In this specific case I agree, since the model may be overfitted, it seems like it's currently just a glorified object classifier based on what was in the training data, but the fact that it works at all may indicate that the underlying idea has merit. They would probably have to train a much larger network to see if it's able to separate features distinctly enough using the input fMRI data to be useful.
It's not an object classifier at all. They had to text-prompt the system, first. I think the general idea is using the fMRI data as the pseudorandom initialization for the latent diffusion model to explore. From what I understand, regular Stable Diffusion starts by generating a noise and then hallucinating modifications of that noise to make less noise. The more you let it run, the better the results. So instead of j…
Briefly reading the paper, it seems they trained 2 models (using data from different stages in the visual cortex) to generate latent vectors for both the visual and textual representations of the fMRA data, then feed those into Stable Diffusion. Those are the models that would be overfit in this case, so instead of those models being able to encode features like "toy, animal, fluffy, brown, ears, nose, arms, legs" individually, it's likely just encoding all of those features combined into a generic "teddy bear" because the input dataset is too small. Obviously this is an oversimplification, but hopefully you get what I mean. I didn't mean it was literally an object classifier, but that the nature of a model like this, with a dataset so small, it does not have to ability to extrapolate fine details. With a larger dataset and more training, it may be able to actually do that.
Re: High-res image reconstruction with latent diffusion models from human brain
#74I immediately found the results suspect, and think I have found what is actually going on. The dataset it was trained on was 2770 images, minus 982 of those used for validation. I posit that the system did not actually read any pictures from the brains, but simply overfitted all the training images into the network itself. For example, if one looks at a picture of a teddy bear, you'd get an overfitted picture of anot…
What are you talking about? They didn't train a model for this. That's why it's so impressive.
Re: High-res image reconstruction with latent diffusion models from human brain
#75here we see, basically, a potential feedback loop. AI tools advance brain science -- more advanced brain science can then inform progress in AI. this is why the situation is dangerous: because people dont think about these feedback loops. people see AI and they move the goalposts and rationalize by saying that "cutting edge AI is still short of AGI so its ok." but most normal people dont think about how AI can be used to create AI or how AI could be used to revolutionize all kinds of fields that then plug back into AI. this is a very dangerous, non-linear space. its not the first non-linear space we have traversed but its certainly the least linear space we have ever entered into and it is the highest stakes humanity has ever or will ever deal with.
even if this is just another bullshit article, im just making a point related to it. people need to be worried about this. for the first time in history, lots of people are now creeped out by AI. but they arent taking action or demanding change. we need regulation, grass-roots efforts to stop AI. even if the only way humanity could abort AI as a concept, or delay it for a significant amount of time, was to return to the iron age, and it certainly isnt the only way, it would be unambiguously worth it, in every way and from every angle.
AI requires large compute. what we are doing now was impossible just 20 years ago. if not 20 then 30. you cant manufacture that kind of compute in your garage. global regulation would take care of it no problem. at the very least it would buy us an enormous amount of time that we could use to figure something else out. people always say that some hold-out country would defy global regulations. they wouldnt defy NATO, let alone a super-global coalition. and the idea of such a group or NATO enforcing compute regulations is not far-fetched whatsoever because the emergence of AGI or even advanced non-AGI goes against the interests of literally every human being. there is no group of humans that benefit from that ultimately. the problem is simply waking people up to this plain fact.
Re: High-res image reconstruction with latent diffusion models from human brain
#76Earlier quoted context omitted.
Many people do have an internal monologue. The vector is that some police unit presents you with a login form (eg. for your password manager or encrypted filesystem), and you involuntarily think of the password, which this device reads and presents to them.
Joke's on them, my passwords are entirely unpronounceable
only my fingers know my passwords. And no way all ten will rat me out
Re: High-res image reconstruction with latent diffusion models from human brain
#77Earlier quoted context omitted.
There was a guy at MIT about ten years ago (edit: 2018! Woah) who made a headset that would read electrical impulses from your face. Apparently when people think in words, the same nerves fire as when they speak, just at a lower activation level. Using those signals it is possible to reconstruct the words being thought. I'm surprised it didn't seem to go anywhere. Edit: found it https://youtu.be/RuUSc53Xpeg
People only think in words right before they say something, so I'm not sure how big a deal this is. I guess they'd be able to predict what I'm writing half a second before I write it? Would be useful if I lost the ability to write or speak, for whatever reason.
I'm "thinking in words" this entire thread as I read it. Do some people read without hearing the words in their head?
Re: High-res image reconstruction with latent diffusion models from human brain
#78Earlier quoted context omitted.
How did you get consent to put the chip in?
Sound's like you're opposed to all animal research, not specifically brain-computer interface research. We also don't ask a monkey's consent before doing any other sort of experiment on it.
I'm not necessarily opposed to it, I simply answered OPs question. It's unethical because there's no consent being given. However, humans do animal research regardless because they view it as "the end justifies the means", and they usually try to use humane methods (which is also not really an excuse because you can't ask the animal if it feels pain, or depression, or anything else).
Re: High-res image reconstruction with latent diffusion models from human brain
#79Earlier quoted context omitted.
People only think in words right before they say something, so I'm not sure how big a deal this is. I guess they'd be able to predict what I'm writing half a second before I write it? Would be useful if I lost the ability to write or speak, for whatever reason.
Citation definitely needed. I have a nearly constant internal monologue that is 100% composed of words.
I only have that when I'm reading something and really trying to take it all in.
Otherwise my internal monologue is a combination of notions, visions, and words.
Do you think in complete sentences?
Re: High-res image reconstruction with latent diffusion models from human brain
#80This is big thing. Although this particular paper is not big thing, the many related quoted studies, set a trend.