Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

High-res image reconstruction with latent diffusion models from human brain

github.com

Post reply on HN
Print view View original

31–40 of 170 posts

Re: High-res image reconstruction with latent diffusion models from human brain

#31

Post by Veen »

zepolen wrote: post #13
tiarafawn wrote: post #10

Earlier quoted context omitted.

The end game here is developing a mind reading device. The endeavor device is ethically questionable because such a device would have a lot of ethically wrong/questionable applications.

The only thing that's ethically questionable are humans themselves and such a device would likely do more to expose the unethical. Everybody remembers what happen when online dna hit mainstream.

Yeah, that's just what we need: brain scanners purported to "expose the unethical". Nothing could go wrong with that.

I find it extremely silly when people argue a technology is value neutral and humans are the problem so we shouldn't "judge" or critique the tech. Who do they think is going to use the technology?

Re: High-res image reconstruction with latent diffusion models from human brain

#32

Post by soulofmischief »

meindnoch wrote: post #25
drzoltar wrote: post #23

My understanding is that we won’t get a “mind reader” model out of this, because visual stimulus vs your imagination happen in separate parts of the brain. In other words we won’t be reading the minds of suspected criminals anytime soon. Maybe someone with neurology experience can chime in here? Is it even theoretically possible to see what’s happening in the imagination?

I have a hunch that any sort of mind-reading machine would have to be tailored uniquely to the individual you want to probe. The internal neural representations likely develop uniquely for each individual.

We'll just train a model on training models.

Re: High-res image reconstruction with latent diffusion models from human brain

#33

Post by wongarsu »

drzoltar wrote: post #23

My understanding is that we won’t get a “mind reader” model out of this, because visual stimulus vs your imagination happen in separate parts of the brain. In other words we won’t be reading the minds of suspected criminals anytime soon. Maybe someone with neurology experience can chime in here? Is it even theoretically possible to see what’s happening in the imagination?

In the best (worst?) case the method generalizes well, and you could just replace the training set of fMRI scans of people viewing images with fMRI scans of people asked to recall images they were shown previously, or fMRI scans of people told to imagine a scene based on a verbal description. It's rarely that easy though

Re: High-res image reconstruction with latent diffusion models from human brain

#34

Post by wongarsu »

soulofmischief wrote: post #32
meindnoch wrote: post #25

Earlier quoted context omitted.

I have a hunch that any sort of mind-reading machine would have to be tailored uniquely to the individual you want to probe. The internal neural representations likely develop uniquely for each individual.

We'll just train a model on training models.

Maybe that's actually feasible, in the sense that you don't have to train a complete new model but can spit out a personalized model based on a calibration sequence of ten images.

Re: High-res image reconstruction with latent diffusion models from human brain

#35

Post by oth001 »

jejeyyy77 wrote: post #9
dheera wrote: post #4

I wonder how well this would work with wearable brainwave detectors rather than MRI, seeing as MRI isn't really something I could have at home.

any idea what the best consumer brainwave detectors are on the market rn?

OpenBCI is one. Not cheap from what I've seen.

Re: High-res image reconstruction with latent diffusion models from human brain

#37

Post by giantrobot »

drzoltar wrote: post #23

My understanding is that we won’t get a “mind reader” model out of this, because visual stimulus vs your imagination happen in separate parts of the brain. In other words we won’t be reading the minds of suspected criminals anytime soon. Maybe someone with neurology experience can chime in here? Is it even theoretically possible to see what’s happening in the imagination?

> In other words we won’t be reading the minds of suspected criminals anytime soon.

Oh don't worry, this will get wrapped up in some pseudoscience bullshit and misleading statistics and marketed to law enforcement. But not to worry, at first it'll only be used on real bad criminals. If you have nothing to hide you have nothing to fear!

Re: High-res image reconstruction with latent diffusion models from human brain

#38

Post by andai »

donohoe wrote: post #29

As people and groups increasingly move this direction do we think about vectors for abuse in 10, 20 or 50+ years? The human mind is considered the only place where we have true privacy. All these efforts are taking that away. At this rate all notions of privacy will soon be dead.

There was a guy at MIT about ten years ago (edit: 2018! Woah) who made a headset that would read electrical impulses from your face. Apparently when people think in words, the same nerves fire as when they speak, just at a lower activation level. Using those signals it is possible to reconstruct the words being thought.

I'm surprised it didn't seem to go anywhere.

Edit: found it https://youtu.be/RuUSc53Xpeg

Re: High-res image reconstruction with latent diffusion models from human brain

#39

Post by Double_a_92 »

Hakkin wrote: post #28
Aransentin wrote: post #15

I immediately found the results suspect, and think I have found what is actually going on. The dataset it was trained on was 2770 images, minus 982 of those used for validation. I posit that the system did not actually read any pictures from the brains, but simply overfitted all the training images into the network itself. For example, if one looks at a picture of a teddy bear, you'd get an overfitted picture of anot…

I'm definitely not an expert in this subject, but even if the model is overfitted, doesn't the fact that it can pull out the similar images at all give credit to the idea that a larger, non-overfitted model could actually work as the paper describes? It means that there does exist some correlation between the shown subject, the captured fMRI data, and the resulting location in latent space.

The output part is basically nonsense. It would be more honest if the output was a text. E.g. "Teddybear" instead of a bad image of a random teddybear.

Re: High-res image reconstruction with latent diffusion models from human brain

#40

Post by burnished »

not-my-account wrote: post #7
rvnx wrote: post #3

Creepy and cool at the same time. It goes into the bucket of things that are not ethically right, same ways as implanting chips to read monkeys brains. But technically interesting and well-executed.

Why is it unethical to put chips in monkey brains?

Well, for one, the lived experience for most science monkeys is "torture, execution, autopsy", and the reason we tamper with their brains is due to the similarity.

I suspect others are taking the wide view but I wanted to point out this direct answer to your question.

p.s the use of "torture" is intended in contrast to the neutrality of clinical language on this topic, not as a hint at my judgement on the matter.

Post reply on HN