More information on this thread: https://twitter.com/ykamit/status/1039672289413844993
Thanks for this!
Deep image reconstruction from human brain activity (2017)
41–50 of 97 posts
Re: Deep image reconstruction from human brain activity (2017)
#42Hmm. Do different humans produce relatively the same brain images for a given scene? If not, I’d imagine training this would be a real PITA.
fMRI data is very noisy, and that's reflected in the generated images. They're not at all recognizable, unless they're sampled from a very small class like simple geometric shapes. I doubt it's possible to do much better, with fMRI.
Re: Deep image reconstruction from human brain activity (2017)
#43Earlier quoted context omitted.
there is also evidence this ability to conjure imagery in our minds simply does not exist for some: https://en.wikipedia.org/wiki/Aphantasia which leads me to wonder if the capability exists on a spectrum and is stronger or weaker in some. A stronger capability might correlate to a stronger 'signal' ...?
Do people actually see images of things in their head? I can close my eyes and think of something, but I won't perceive anything more than a linear description of it. At best, I can use these visual descriptions to draw an outline in my head, but it instantly fades out if I don't continuously re-draw it. Kind of like this: https://i.imgur.com/0zuPIPV.gifv
Supposedly, most people do. There was an article on hn about "aphantasia" (the lack of mental imagery) not to long ago, but I can't seem to find it now. Google turns up quite a few hits though.
I can't help but be a little sceptical of the concept though. I believe there's probably a spectrum but maybe not as wide is claimed. Based on the comment sections of basically any article on the subect, I wonder if there isn't something like imposter syndrome going on. Where the majority of people dont think they have as sharp mental imagery as the average person.
Re: Deep image reconstruction from human brain activity (2017)
#44Earlier quoted context omitted.
Do people actually see images of things in their head? I can close my eyes and think of something, but I won't perceive anything more than a linear description of it. At best, I can use these visual descriptions to draw an outline in my head, but it instantly fades out if I don't continuously re-draw it. Kind of like this: https://i.imgur.com/0zuPIPV.gifv
I have no issue conjuring clear images when closing eyes. I guess it depends how your brain is wired and this is probably why we have ways to refer people who are good at drawing, painting, etc. They probably have a mix of clear visualization and very good hand/finger coordination.
- Can you imagine the color of the floor, walls and ceiling at work?
- Can you imagine the color of the eyes and hairs of your coworkers?
- Can you imagine the home screen of your phone and see how many application icons can fit in a row and column?
- Can you imagine Google's logo and see the color of each letter?
- Can you imagine your car manufacturer's logo and draw it?
Re: Deep image reconstruction from human brain activity (2017)
#45This is impressive, and certainly technology like this will probably find lots of very good uses, but I can't help imagining(!) dystopian societies where "thoughtcrime" can actually be monitored and "made actionable".
A world in which "you have nothing to hide" turns into "you have nothing you can hide" is immensely disturbing.
Re: Deep image reconstruction from human brain activity (2017)
#46Earlier quoted context omitted.
there is also evidence this ability to conjure imagery in our minds simply does not exist for some: https://en.wikipedia.org/wiki/Aphantasia which leads me to wonder if the capability exists on a spectrum and is stronger or weaker in some. A stronger capability might correlate to a stronger 'signal' ...?
Do people actually see images of things in their head? I can close my eyes and think of something, but I won't perceive anything more than a linear description of it. At best, I can use these visual descriptions to draw an outline in my head, but it instantly fades out if I don't continuously re-draw it. Kind of like this: https://i.imgur.com/0zuPIPV.gifv
Can't reply to your other comment, presumably due to comment depth -- but yes, I did all of those vividly and with ease, even with my eyes open, except for counting the number of icons in my imagined phone screen. Somehow I can imagine the phone screen as a whole, filled with icons, but actually counting them feels weird, like how reading written text inside a dream is weird.
Until I read [0], I had never thought about it and just assumed that everything else was like me. Hah!
[0] https://www.lesswrong.com/posts/baTWMegR42PAsH9qJ/generalizi...
Re: Deep image reconstruction from human brain activity (2017)
#47Acknowledging that I know nothing about psychology or neuroscience... It's interesting that they were not able to extract good images from the test when they just asked the subjects to imagine the pictures. I speculate that is because the vast majority of people are able to recognize images of things, but not produce those images. For a simple example, see this article about people trying to draw a bicycle. Before cl…
Feynman wrote about this in one of his books. it's really hard to understand how someone else understands something. but you can say, well imagine it's orange and spherical and kind of hairy, and they'll have a picture in their mind you can kinda work with.
Are you sure you're not thinking of the "hairy green ball thing" from his discussion of visualizing topology examples?
https://books.google.com/books?id=_gA_DwAAQBAJ&pg=PT87&lpg=P...
Re: Deep image reconstruction from human brain activity (2017)
#48When we build neural networks to decode our own neural networks...
Do you remember that project that mapped where words are "stored" in the brain? I don't have it right now, but it was something like that. They used a neural approach to map the word vectors in the brain.
That sounds like the brain atlas created with subjects listening to stories in an fMRI machine.
Here's a Guardian article about it: https://www.theguardian.com/science/2016/apr/27/brain-atlas-...
And a link to the paper: https://www.nature.com/articles/nature17637
Re: Deep image reconstruction from human brain activity (2017)
#49Wow, I'm calling a hoax. They produce images of dubious value, and for whatever reason, contaminate the results by mutating them with the Deep Dream stylization. This is as bad as the dead salmon results. EDIT: You can flag all you want, but look at this playback of the reconstruction: https://video.twimg.com/tweet_video/DSrWxhSVQAE2Xix.mp4 There's nothing useful happening here.
The code and data is here: https://github.com/KamitaniLab/DeepImageReconstruction Not a hoax. Your comparison to Deep Dream is quite apt, for good reason that you clearly don't realize- Deep Dream is an attempt to "enhance patterns in images via algorithmic pareidolia"- Pareidolia being a "psychological phenomenon in which the mind responds to a stimulus". So yes, this project is affected by a real life "deep dream".
Oh, a github repo suddenly validates scientific claims. The presence of an open repo really doesn't bolster the claims of this project.
As if to say "go ahead, try it yourself. you can replicate your own results."
That doesn't improve the premise of this junk science, since the images are fabricated, and poor quality besides.
Re: Deep image reconstruction from human brain activity (2017)
#50Wow, I'm calling a hoax. They produce images of dubious value, and for whatever reason, contaminate the results by mutating them with the Deep Dream stylization. This is as bad as the dead salmon results. EDIT: You can flag all you want, but look at this playback of the reconstruction: https://video.twimg.com/tweet_video/DSrWxhSVQAE2Xix.mp4 There's nothing useful happening here.
That's exactly what they say they're doing. That's not "contamination" - that's the whole point.
And that's exactly why this is garbage research. Point deep dream at an interference pattern of pure noise, after training it on a set of target images.
Don't be surprised when it "finds" the image, pulled from raw static that contained no signal.