Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep-learning algorithm predicts photos’ memorability at “near-human” levels

news.mit.edu

Post reply on HN
Print view View original

11–20 of 26 posts

Re: Deep-learning algorithm predicts photos’ memorability at “near-human” levels

#11

Post by danso »

arethuza wrote: post #10
nicklo wrote: post #3

The applications for this in the advertising industry are exciting and a tad bit scary. I can imagine a feedback system that takes the output of this algorithm and modifies the image slightly according to the error gradient to optimize the MEM-score of the brand/item being advertised in the photo. Then it could feed the new image back in and repeat like the Deep-Dreaming algorithm. side-note: I'm so happy that CSAIL…

In another thread, someone mentioned a mobile app that paused ad videos if you weren't looking at the device. Maybe the approach you mention could be applied in a similar way - to morph images in ads until you do look at them.

This problem has been mostly solved. You don't need a deep learning algorithm to detect a face. There's an entire company that has sprouted from online education by providing solutions for the proctoring of exams (the webcam is turned on, and facial movement that suspiciously turns towards offscreen areas is flagged). I'm not even sure the phone's camera would have to be on to do the mobile app detection you describe.

Re: Deep-learning algorithm predicts photos’ memorability at “near-human” levels

#14

Post by ioeu »

> While deep-learning has propelled much progress in object recognition and scene understanding, predicting human memory has often been viewed as a higher-level cognitive process that computer scientists will never be able to tackle

Seriously?

Has it seriously often been viewed like that?

Re: Deep-learning algorithm predicts photos’ memorability at “near-human” levels

#15

Post by [deleted] »

ioeu wrote: post #14

> While deep-learning has propelled much progress in object recognition and scene understanding, predicting human memory has often been viewed as a higher-level cognitive process that computer scientists will never be able to tackle Seriously? Has it seriously often been viewed like that?

[deleted]

Re: Deep-learning algorithm predicts photos’ memorability at “near-human” levels

#16

Post by ioeu »

> For each image, the algorithm produces a heat map showing which parts of the image are most memorable. By emphasizing different regions, they can potentially increase the image’s memorability.

Most memorable, according to human subjects subjective thought on the matter?

> The team then pitted its algorithm against human subjects by having the model predicting how memorable a group of people would find a new never-before-seen image. It performed 30 percent better than existing algorithms and was within a few percentage points of the average human performance.

Who's to say human subjects are any good at objectively judging how memorable a photo is? I feel like I'm missing something.

Edit: Riight, I guess it could be based on observing neural activity in human subjects while they look at photos. That makes a lot more sense.

Re: Deep-learning algorithm predicts photos’ memorability at “near-human” levels

#17

Post by pseud0r »

Sounds like maybe this could be used for deep-learning networks to learn what to pay attention to. The memorable regions should be the salient or important regions in the image. Basically the more you are paying attention, the more memorable something is.

Re: Deep-learning algorithm predicts photos’ memorability at “near-human” levels

#18

Post by florianletsch »

ioeu wrote: post #16

> For each image, the algorithm produces a heat map showing which parts of the image are most memorable. By emphasizing different regions, they can potentially increase the image’s memorability. Most memorable, according to human subjects subjective thought on the matter? > The team then pitted its algorithm against human subjects by having the model predicting how memorable a group of people would find a new never-b…

Not based on human subject's opinion but on measured performance:

> The images had each received a “memorability score” based on the ability of human subjects to remember them in online experiments.

Re: Deep-learning algorithm predicts photos’ memorability at “near-human” levels

#19

Post by arethuza »

danso wrote: post #11
arethuza wrote: post #10

Earlier quoted context omitted.

In another thread, someone mentioned a mobile app that paused ad videos if you weren't looking at the device. Maybe the approach you mention could be applied in a similar way - to morph images in ads until you do look at them.

This problem has been mostly solved. You don't need a deep learning algorithm to detect a face. There's an entire company that has sprouted from online education by providing solutions for the proctoring of exams (the webcam is turned on, and facial movement that suspiciously turns towards offscreen areas is flagged). I'm not even sure the phone's camera would have to be on to do the mobile app detection you describe…

That's what I meant - the application that detect whether you are looking at an ad is already out there. What I meant was that rather than simply pausing the video it could change the image iteratively until you do look at the image.

Re: Deep-learning algorithm predicts photos’ memorability at “near-human” levels

#20

Post by stkni »

Uploading a URL of white noise from Wikipedia gave it a high-memorability (0.82) with lots of areas of interest. Secondly uploading a purely white image [2] produced high areas of interest in the top corners and a mediumly interesting image (0.62).

I thought the tests might reveal something useful, like the eye-tracking heat-maps of Jakob Nielsen [3] but I'm not convinced.

[1] https://upload.wikimedia.org/wikipedia/commons/f/f6/White-no...

[2] http://images.all-free-download.com/images/graphiclarge/plai...

[3] https://www.nngroup.com/books/eyetracking-web-usability/

Post reply on HN