Deep-learning algorithm predicts photos’ memorability at “near-human” levels
1–10 of 26 posts
Re: Deep-learning algorithm predicts photos’ memorability at “near-human” levels
#2Re: Deep-learning algorithm predicts photos’ memorability at “near-human” levels
#3The applications for this in the advertising industry are exciting and a tad bit scary.
I can imagine a feedback system that takes the output of this algorithm and modifies the image slightly according to the error gradient to optimize the MEM-score of the brand/item being advertised in the photo. Then it could feed the new image back in and repeat like the Deep-Dreaming algorithm.
side-note: I'm so happy that CSAIL is finally embracing deep learning. I'm an undergrad at mit and this semester was the first that deep learning was a major part of both the computer vision class and the nlp class.
Re: Deep-learning algorithm predicts photos’ memorability at “near-human” levels
#4The point they seem to be going out of their way to avoid is whether humans are any good at this task. Anyone know if they are?
Re: Deep-learning algorithm predicts photos’ memorability at “near-human” levels
#5The applications for this in the advertising industry are exciting and a tad bit scary. I can imagine a feedback system that takes the output of this algorithm and modifies the image slightly according to the error gradient to optimize the MEM-score of the brand/item being advertised in the photo. Then it could feed the new image back in and repeat like the Deep-Dreaming algorithm. side-note: I'm so happy that CSAIL…
Although your idea is absolutely terrifying, I could see it having some pretty entertaining failure modes.
For example: a sportswear company uses an image destined for their Instagram feed as input. It's a photo of a famous athlete touting their product. After they process the image, the output has a higher MEM-score—but only because the athlete is now dressed in the attire of their competitors.
Re: Deep-learning algorithm predicts photos’ memorability at “near-human” levels
#6I don't see this being of huge use to most media campaigns in which a human editor is involved...though perhaps it could be one of several first-pass filters used to go through a digital photographer's memory card to and filter out the weakest images (personally, as a photographer, a much simpler tool that can weed out obviously blurry or unfocused images would be much, much, much more useful than something that tells me what I hope I already know and enjoy doing: picking out my favorite photos)
However, this algorithm would be immediately useful for people who need to auto crop photos in a way more intelligently than "just fit this dimension and ratio"...but this function has been implemented to some degree by various other computer vision systems, such as Microsoft's Projext Oxford https://www.projectoxford.ai/vision
Re: Deep-learning algorithm predicts photos’ memorability at “near-human” levels
#7These are interesting bits: http://memorability.csail.mit.edu/download.html
https://people.csail.mit.edu/khosla/papers/iccv2015_khosla.p...
Re: Deep-learning algorithm predicts photos’ memorability at “near-human” levels
#8Also https://news.ycombinator.com/item?id=10747490 .
There's an interesting tidbit there:
> The latest version of MemNet is available online. Being an amateur cat photographer myself, I decided to give this a try. Apparently, the most memorable part of Mr. Tango Tangerine’s face is his left ear
The cat photo is pretty ordinary, as far as photos of cats taken by their loving owners go...though I could imagine why the algorithm behaved the way it did, id be interested in hearing anyone try to argue that the algorithm picked something remotely relevant to the human experience. I mean, if the ear were deformed or on fire, sure...but it's not interesting in any way, even if you take the tack of "we'll all cats look the same anyway so no one will remember the cats face"
That said, a huge kudos to the MIT researchers for not only open sourcing their work, but releasing a straightforward REST API to make it easy for anyone to test out their algorithm.
Re: Deep-learning algorithm predicts photos’ memorability at “near-human” levels
#9Maybe all the world is uploading pictures right now but after several minutes the site is still "Computing..." the one I uploaded. It's only me? I tried with Firefox and Opera.
Re: Deep-learning algorithm predicts photos’ memorability at “near-human” levels
#10The applications for this in the advertising industry are exciting and a tad bit scary. I can imagine a feedback system that takes the output of this algorithm and modifies the image slightly according to the error gradient to optimize the MEM-score of the brand/item being advertised in the photo. Then it could feed the new image back in and repeat like the Deep-Dreaming algorithm. side-note: I'm so happy that CSAIL…
In another thread, someone mentioned a mobile app that paused ad videos if you weren't looking at the device. Maybe the approach you mention could be applied in a similar way - to morph images in ads until you do look at them.