Live data from Hacker News

High-res image reconstruction with latent diffusion models from human brain

github.com

61–70 of 170 posts

Re: High-res image reconstruction with latent diffusion models from human brain

#61
post #41
post #14

Earlier quoted context omitted.

How did you get consent to put the chip in?

Sound's like you're opposed to all animal research, not specifically brain-computer interface research. We also don't ask a monkey's consent before doing any other sort of experiment on it.

[deleted]

Re: High-res image reconstruction with latent diffusion models from human brain

#62
post #38

Earlier quoted context omitted.

There was a guy at MIT about ten years ago (edit: 2018! Woah) who made a headset that would read electrical impulses from your face. Apparently when people think in words, the same nerves fire as when they speak, just at a lower activation level. Using those signals it is possible to reconstruct the words being thought. I'm surprised it didn't seem to go anywhere. Edit: found it https://youtu.be/RuUSc53Xpeg

People only think in words right before they say something, so I'm not sure how big a deal this is. I guess they'd be able to predict what I'm writing half a second before I write it? Would be useful if I lost the ability to write or speak, for whatever reason.

The extent that people's thinking relies on inner monologue is something that varies wildly between different people. Likewise people's abilities to form mental images.

Re: High-res image reconstruction with latent diffusion models from human brain

#63

Earlier quoted context omitted.

People only think in words right before they say something, so I'm not sure how big a deal this is. I guess they'd be able to predict what I'm writing half a second before I write it? Would be useful if I lost the ability to write or speak, for whatever reason.

Many people do have an internal monologue. The vector is that some police unit presents you with a login form (eg. for your password manager or encrypted filesystem), and you involuntarily think of the password, which this device reads and presents to them.

Joke's on them, my passwords are entirely unpronounceable

Re: High-res image reconstruction with latent diffusion models from human brain

#64
post #4

I wonder how well this would work with wearable brainwave detectors rather than MRI, seeing as MRI isn't really something I could have at home.

By brainwave detector I am going to assume you mean an EEG. An EEG measures electrical activity at the surface of the brain. A fMRI shows the activity of individual neurons of the entire brain in real time. It's sort of an apples to oranges comparison given the tools measure different things at vastly different resolutions.

Re: High-res image reconstruction with latent diffusion models from human brain

#66
post #29

As people and groups increasingly move this direction do we think about vectors for abuse in 10, 20 or 50+ years? The human mind is considered the only place where we have true privacy. All these efforts are taking that away. At this rate all notions of privacy will soon be dead.

This is something 1984 was slightly hinting at.

Of course, these "advances" will be praised greatly in MSM as providing great benefits for mutes, "harmonious society", and whatever else happens to be the virtue-signaling fad of the moment.

Re: High-res image reconstruction with latent diffusion models from human brain

#67
post #28

I immediately found the results suspect, and think I have found what is actually going on. The dataset it was trained on was 2770 images, minus 982 of those used for validation. I posit that the system did not actually read any pictures from the brains, but simply overfitted all the training images into the network itself. For example, if one looks at a picture of a teddy bear, you'd get an overfitted picture of anot…

I'm definitely not an expert in this subject, but even if the model is overfitted, doesn't the fact that it can pull out the similar images at all give credit to the idea that a larger, non-overfitted model could actually work as the paper describes? It means that there does exist some correlation between the shown subject, the captured fMRI data, and the resulting location in latent space.

Nope.

If you train a model where the input is an integer between 1 and 10, and the output is a specific image from a set of ten, the model will be able to get zero loss on the task. That is what's happening here.

Re: High-res image reconstruction with latent diffusion models from human brain

#68
post #16
post #14

Earlier quoted context omitted.

How did you get consent to put the chip in?

You ask for consent through the brain-computer interface. /s

Conclusion from scientists: "If monkey drinks the smoothie, then it means he wants a surgery."

Re: High-res image reconstruction with latent diffusion models from human brain

#69
post #46

Earlier quoted context omitted.

In this specific case I agree, since the model may be overfitted, it seems like it's currently just a glorified object classifier based on what was in the training data, but the fact that it works at all may indicate that the underlying idea has merit. They would probably have to train a much larger network to see if it's able to separate features distinctly enough using the input fMRI data to be useful.

The problem is that it's impossible to know what is in the fMRI data and what is hallucinated by the reconstruction. In this case, the real bear has a blue ribbon and the "reconstructed" bear ha a red ribbon. Is the ribbon in the fMRI data and the computer choose the wrong color, or most of the images in the training set had ribbons and the computer just added one. Imagine this something like this is used in the futu…

[deleted]

Re: High-res image reconstruction with latent diffusion models from human brain

#70

I immediately found the results suspect, and think I have found what is actually going on. The dataset it was trained on was 2770 images, minus 982 of those used for validation. I posit that the system did not actually read any pictures from the brains, but simply overfitted all the training images into the network itself. For example, if one looks at a picture of a teddy bear, you'd get an overfitted picture of anot…

there is also no way that you could represent details as shown with such a small sample.
Post reply on HN