Live data from Hacker News

High-res image reconstruction with latent diffusion models from human brain

github.com

41–50 of 170 posts

Re: High-res image reconstruction with latent diffusion models from human brain

#41
post #14

Earlier quoted context omitted.

Why is it unethical to put chips in monkey brains?

How did you get consent to put the chip in?

Sound's like you're opposed to all animal research, not specifically brain-computer interface research. We also don't ask a monkey's consent before doing any other sort of experiment on it.

Re: High-res image reconstruction with latent diffusion models from human brain

#42

I immediately found the results suspect, and think I have found what is actually going on. The dataset it was trained on was 2770 images, minus 982 of those used for validation. I posit that the system did not actually read any pictures from the brains, but simply overfitted all the training images into the network itself. For example, if one looks at a picture of a teddy bear, you'd get an overfitted picture of anot…

I feel like you might be moving the goal posts here a bit. Getting a reconstruction that is a bear, even if not the same bear, is impressive enough to be noteworthy.

I think the point is that it's not a reconstruction. It's more like recognizing which letter of a thousand-letter alphabet is shown to the human after decoding their brain waves. Still impressive, but not really as impressive as visual reconstruction.

Re: High-res image reconstruction with latent diffusion models from human brain

#43

Are any of the example images novel, i.e. new to the model? Or is the model only reconstructing images it has already seen before? Either way, if I'm understanding right, it's very impressive. If the only input to the model (after training) is a fMRI reading, and from that it can reconstruct an image, at the very least that shows it can strongly correlate brain patterns back to the original image. It'd be even cooler…

There was a video many years ago (early 2010s?) demoing a similar technology, which would overlay and blend many images on top of each other to make a fuzzy image approximating what was actually being viewed.

Edit: found it! https://youtu.be/nsjDnYxJ0bo

Re: High-res image reconstruction with latent diffusion models from human brain

#44
post #29

As people and groups increasingly move this direction do we think about vectors for abuse in 10, 20 or 50+ years? The human mind is considered the only place where we have true privacy. All these efforts are taking that away. At this rate all notions of privacy will soon be dead.

> As people and groups increasingly move this direction do we think about vectors for abuse in 10, 20 or 50+ years?

No, the delusional shortsighted and revenue-driven SV startup culture doesn't give a shit about such 'technophobic trivialities'.

Re: High-res image reconstruction with latent diffusion models from human brain

#45
post #29

As people and groups increasingly move this direction do we think about vectors for abuse in 10, 20 or 50+ years? The human mind is considered the only place where we have true privacy. All these efforts are taking that away. At this rate all notions of privacy will soon be dead.

There is an amazing novel called The Truth Machine by James L Halperin that speaks about this.

Re: High-res image reconstruction with latent diffusion models from human brain

#46
post #28

Earlier quoted context omitted.

I'm definitely not an expert in this subject, but even if the model is overfitted, doesn't the fact that it can pull out the similar images at all give credit to the idea that a larger, non-overfitted model could actually work as the paper describes? It means that there does exist some correlation between the shown subject, the captured fMRI data, and the resulting location in latent space.

The output part is basically nonsense. It would be more honest if the output was a text. E.g. "Teddybear" instead of a bad image of a random teddybear.

In this specific case I agree, since the model may be overfitted, it seems like it's currently just a glorified object classifier based on what was in the training data, but the fact that it works at all may indicate that the underlying idea has merit. They would probably have to train a much larger network to see if it's able to separate features distinctly enough using the input fMRI data to be useful.

Re: High-res image reconstruction with latent diffusion models from human brain

#47

Earlier quoted context omitted.

We'll just train a model on training models.

Maybe that's actually feasible, in the sense that you don't have to train a complete new model but can spit out a personalized model based on a calibration sequence of ten images.

Yep! This process is known as fine-tuning.

Re: High-res image reconstruction with latent diffusion models from human brain

#48
post #29

As people and groups increasingly move this direction do we think about vectors for abuse in 10, 20 or 50+ years? The human mind is considered the only place where we have true privacy. All these efforts are taking that away. At this rate all notions of privacy will soon be dead.

A decade ago we had https://youtu.be/nsjDnYxJ0bo

Re: High-res image reconstruction with latent diffusion models from human brain

#49
post #29

As people and groups increasingly move this direction do we think about vectors for abuse in 10, 20 or 50+ years? The human mind is considered the only place where we have true privacy. All these efforts are taking that away. At this rate all notions of privacy will soon be dead.

If this technology becomes accessible to courtrooms or police, they will use it. There will never be a way to encrypt thoughts.

Re: High-res image reconstruction with latent diffusion models from human brain

#50
post #38
post #29

As people and groups increasingly move this direction do we think about vectors for abuse in 10, 20 or 50+ years? The human mind is considered the only place where we have true privacy. All these efforts are taking that away. At this rate all notions of privacy will soon be dead.

There was a guy at MIT about ten years ago (edit: 2018! Woah) who made a headset that would read electrical impulses from your face. Apparently when people think in words, the same nerves fire as when they speak, just at a lower activation level. Using those signals it is possible to reconstruct the words being thought. I'm surprised it didn't seem to go anywhere. Edit: found it https://youtu.be/RuUSc53Xpeg

> I'm surprised it didn't seem to go anywhere.

At least not publicly.

Post reply on HN