Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

High-res image reconstruction with latent diffusion models from human brain

github.com

Post reply on HN
Print view View original

21–30 of 170 posts

Re: High-res image reconstruction with latent diffusion models from human brain

#21

Post by kdma »

Aransentin wrote: post #15

I immediately found the results suspect, and think I have found what is actually going on. The dataset it was trained on was 2770 images, minus 982 of those used for validation. I posit that the system did not actually read any pictures from the brains, but simply overfitted all the training images into the network itself. For example, if one looks at a picture of a teddy bear, you'd get an overfitted picture of anot…

Good find, when I read it I called bullshit but I got lost trying to understand the diagrams. Another gotcha is the semantic decoder, they are just looping the model on itself "A cozy teddy bear" + fMRI random input => A teddy bear!!!

Re: High-res image reconstruction with latent diffusion models from human brain

#22

Post by nhchris »

zepolen wrote: post #13
tiarafawn wrote: post #10

Earlier quoted context omitted.

The end game here is developing a mind reading device. The endeavor device is ethically questionable because such a device would have a lot of ethically wrong/questionable applications.

The only thing that's ethically questionable are humans themselves and such a device would likely do more to expose the unethical. Everybody remembers what happen when online dna hit mainstream.

I think only good thoughts so I'm not worried. Plus anyone with control of this technology is sure to have our best interests in mind.

Re: High-res image reconstruction with latent diffusion models from human brain

#23

Post by drzoltar »

My understanding is that we won’t get a “mind reader” model out of this, because visual stimulus vs your imagination happen in separate parts of the brain. In other words we won’t be reading the minds of suspected criminals anytime soon. Maybe someone with neurology experience can chime in here? Is it even theoretically possible to see what’s happening in the imagination?

Re: High-res image reconstruction with latent diffusion models from human brain

#24

Post by SubiculumCode »

ninesnines wrote: post #18

I am suspicious of these results; if we blast a high frequency visual stimulus of a couple of letters and do quite a lot of post processing we can sometimes get a visual cortex map of those particular letters. However, these paper examples are very complex images and I’m very doubtful of the results - aransentin above made a couple of very valid points

I mean in 2011, they were already able to do some basic reconstruction: see https://www.youtube.com/watch?v=nsjDnYxJ0bo

I could imagine improvements since then,especially with advances in image networks.

Re: High-res image reconstruction with latent diffusion models from human brain

#25

Post by meindnoch »

drzoltar wrote: post #23

My understanding is that we won’t get a “mind reader” model out of this, because visual stimulus vs your imagination happen in separate parts of the brain. In other words we won’t be reading the minds of suspected criminals anytime soon. Maybe someone with neurology experience can chime in here? Is it even theoretically possible to see what’s happening in the imagination?

I have a hunch that any sort of mind-reading machine would have to be tailored uniquely to the individual you want to probe. The internal neural representations likely develop uniquely for each individual.

Re: High-res image reconstruction with latent diffusion models from human brain

#26

Post by SubiculumCode »

Aransentin wrote: post #15

I immediately found the results suspect, and think I have found what is actually going on. The dataset it was trained on was 2770 images, minus 982 of those used for validation. I posit that the system did not actually read any pictures from the brains, but simply overfitted all the training images into the network itself. For example, if one looks at a picture of a teddy bear, you'd get an overfitted picture of anot…

I feel like you might be moving the goal posts here a bit. Getting a reconstruction that is a bear, even if not the same bear, is impressive enough to be noteworthy.

Re: High-res image reconstruction with latent diffusion models from human brain

#27

Post by bawolff »

Aransentin wrote: post #15

I immediately found the results suspect, and think I have found what is actually going on. The dataset it was trained on was 2770 images, minus 982 of those used for validation. I posit that the system did not actually read any pictures from the brains, but simply overfitted all the training images into the network itself. For example, if one looks at a picture of a teddy bear, you'd get an overfitted picture of anot…

Even if true, the result still seems very impressive to me as a layman.

Re: High-res image reconstruction with latent diffusion models from human brain

#28

Post by Hakkin »

Aransentin wrote: post #15

I immediately found the results suspect, and think I have found what is actually going on. The dataset it was trained on was 2770 images, minus 982 of those used for validation. I posit that the system did not actually read any pictures from the brains, but simply overfitted all the training images into the network itself. For example, if one looks at a picture of a teddy bear, you'd get an overfitted picture of anot…

I'm definitely not an expert in this subject, but even if the model is overfitted, doesn't the fact that it can pull out the similar images at all give credit to the idea that a larger, non-overfitted model could actually work as the paper describes? It means that there does exist some correlation between the shown subject, the captured fMRI data, and the resulting location in latent space.

Re: High-res image reconstruction with latent diffusion models from human brain

#29

Post by donohoe »

As people and groups increasingly move this direction do we think about vectors for abuse in 10, 20 or 50+ years?

The human mind is considered the only place where we have true privacy. All these efforts are taking that away.

At this rate all notions of privacy will soon be dead.

Re: High-res image reconstruction with latent diffusion models from human brain

#30

Post by arnarbi »

Aransentin wrote: post #15

I immediately found the results suspect, and think I have found what is actually going on. The dataset it was trained on was 2770 images, minus 982 of those used for validation. I posit that the system did not actually read any pictures from the brains, but simply overfitted all the training images into the network itself. For example, if one looks at a picture of a teddy bear, you'd get an overfitted picture of anot…

Subject 4 in the first line also looks very different from the ground truth, but clearly an airliner. I'm curious if there is also a closer match to that one in the set.
Post reply on HN