Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Improving YouTube video thumbnails with deep neural nets

youtube-eng.blogspot.com

Post reply on HN
Print view View original

11–20 of 26 posts

Re: Improving YouTube video thumbnails with deep neural nets

#11

Post by JoshTriplett »

gizmo686 wrote: post #4
Scaevolus wrote: post #2

I wonder if they included sexy images in their negative training sets -- many videos accrue millions of views (and ad dollars) by having a few frames of cleavage interspersed with other (often derivative) footage. It would be great if their algorithm picked a thumbnail that reflected the entire video, not just a few frames specifically chosen to game people's compulsive clicking.

They stated that the negative training set was constructed by randomly sampling frames from the video. If someone wants to game the thumbnails, then they will just manually select the thumbnail to use; and there are to many legitimate use cases for this ability for Youtube to remove it.

> If someone wants to game the thumbnails, then they will just manually select the thumbnail to use; and there are to many legitimate use cases for this ability for Youtube to remove it.

Many channels I watch carefully select an iconic frame from the video to serve as the thumbnail, or construct an artificial thumbnail that provides useful information about the type and subject of the video. Manual will frequently produce better results than automatic for a good-quality channel.

Re: Improving YouTube video thumbnails with deep neural nets

#12

Post by JoshTriplett »

nacs wrote: post #8
pupulon9 wrote: post #6

Earlier quoted context omitted.

Is there a way YouTube could alter the "view count" to only include views where 100% of the video has been watched? May help cut down on videos with misleading thumbnails and/or titles.

A better way to filter those out algorithmically would be to simply look at the thumbs-up vs thumbs-down ratio. The ones with misleading titles/thumbnails often have far more down-votes than up-votes yet YouTube continues to show those as the highest recommended/relevant (I guess Google prefers click-throughs over user-satisfaction).

> A better way to filter those out algorithmically would be to simply look at the thumbs-up vs thumbs-down ratio.

> The ones with misleading titles/thumbnails often have far more down-votes than up-votes

Especially once the total votes pass a certain threshold. Below a certain threshold, any activity makes something interesting; you wouldn't want to let a handful of downvotes bury something early on (as in, 4 upvotes and 6 downvotes). But once you hit the hundreds or thousands of votes, the ratio should take over.

Re: Improving YouTube video thumbnails with deep neural nets

#13

Post by JoshTriplett »

nacs wrote: post #7
Scaevolus wrote: post #2

I wonder if they included sexy images in their negative training sets -- many videos accrue millions of views (and ad dollars) by having a few frames of cleavage interspersed with other (often derivative) footage. It would be great if their algorithm picked a thumbnail that reflected the entire video, not just a few frames specifically chosen to game people's compulsive clicking.

Most of those are just manually selected thumbnails by the uploader. After uploading, YT gives you 3-5 thumbnails you can choose from. Also, partnered accounts are allowed to upload custom thumbnails (which can be any image, not necessarily even a screenshot from the video).

> After uploading, YT gives you 3-5 thumbnails you can choose from.

Can you pick an arbitrary video frame, or only one of the suggested thumbnails?

Re: Improving YouTube video thumbnails with deep neural nets

#14

Post by nacs »

JoshTriplett wrote: post #13
nacs wrote: post #7

Earlier quoted context omitted.

Most of those are just manually selected thumbnails by the uploader. After uploading, YT gives you 3-5 thumbnails you can choose from. Also, partnered accounts are allowed to upload custom thumbnails (which can be any image, not necessarily even a screenshot from the video).

> After uploading, YT gives you 3-5 thumbnails you can choose from. Can you pick an arbitrary video frame, or only one of the suggested thumbnails?

It automatically captures 3 different thumbnails (I guess using the algorithm in OP) and lets you select any 1 of those 3.

Re: Improving YouTube video thumbnails with deep neural nets

#15

Post by chadzawistowski »

JoshTriplett wrote: post #13
nacs wrote: post #7

Earlier quoted context omitted.

Most of those are just manually selected thumbnails by the uploader. After uploading, YT gives you 3-5 thumbnails you can choose from. Also, partnered accounts are allowed to upload custom thumbnails (which can be any image, not necessarily even a screenshot from the video).

> After uploading, YT gives you 3-5 thumbnails you can choose from. Can you pick an arbitrary video frame, or only one of the suggested thumbnails?

Many videos seem to have completely arbitrary thumbnails which are not from the video. Most of the Epic Rap Battles videos, for example.

Perhaps this option 'unlocks' after you reach a certain subscriber count.

Re: Improving YouTube video thumbnails with deep neural nets

#16

Post by mutagen »

They've got to be training on more inputs than mentioned. For example, is one or a close set of times in the video linked externally and generating traffic? Grab the entire set of frames from that time period and run it through the quality classifier, there might be iconic frames from that section that people are looking for.

Are people re-watching a small segment of the video? Try classifying individual frames from that segment or just before. Of course, those are often action moments that result in smeared motion and artifacts and may not result in a quality thumbnail.

These ideas also only come into play when a video has been live for a while, after the uploader has initially picked a thumbnail. Maybe a "We have some new thumbnail suggestions for you, take a look" alert or message?

Re: Improving YouTube video thumbnails with deep neural nets

#17

Post by JoshTriplett »

chadzawistowski wrote: post #15
JoshTriplett wrote: post #13

Earlier quoted context omitted.

> After uploading, YT gives you 3-5 thumbnails you can choose from. Can you pick an arbitrary video frame, or only one of the suggested thumbnails?

Many videos seem to have completely arbitrary thumbnails which are not from the video. Most of the Epic Rap Battles videos, for example. Perhaps this option 'unlocks' after you reach a certain subscriber count.

As mentioned upthread:

> partnered accounts are allowed to upload custom thumbnails (which can be any image, not necessarily even a screenshot from the video).

Re: Improving YouTube video thumbnails with deep neural nets

#18

Post by needBigrPics »

So, in an article about image processing, why not include nice big beautiful images, that get even bigger when you click on them?

I click on the low detail inline images, and they stay the same disappointing size and reveal no further detail.

They're all, like 600px X 200px? Am I being greedy for want of gigantic images, upwards of 3000px wide?

I suppose it is an article about thumbnails, after all, so maybe I shouldn't be so surprised.

Re: Improving YouTube video thumbnails with deep neural nets

#19

Post by dredmorbius »

nacs wrote: post #8
pupulon9 wrote: post #6

Earlier quoted context omitted.

Is there a way YouTube could alter the "view count" to only include views where 100% of the video has been watched? May help cut down on videos with misleading thumbnails and/or titles.

A better way to filter those out algorithmically would be to simply look at the thumbs-up vs thumbs-down ratio. The ones with misleading titles/thumbnails often have far more down-votes than up-votes yet YouTube continues to show those as the highest recommended/relevant (I guess Google prefers click-throughs over user-satisfaction).

Both explicit (thumbs up/down) and implicit (click-aways / closing window) may count toward quality.

There are other confusing cases. I watch a lot of long-form videos, some too long to view in a single session, many of which I download for offline viewing (yt-download). I've been quite actively dissuaded from either publicly rating videos, or even linking to YouTube itself on my primary social channel (G+) given the Anschluss forced-marriage between YouTube, G+, and what had once been individual and separate accounts (similar logic applies to Google Play, and I've taken to "registering" my Android devices under randomly generated usernames).

For videos I particularly like, I may reference them, but only specific portions which I skip to, view, and then close. That's far less than a 100% view, but still significant.

It's not that I'm opposed to providing appropriateness and quality data to YouTube. I absolutely give massive shits about who they share that data with, and how. The "make it all public" default is utterly fucked in the head.

I think Google are starting to realise that.

Post reply on HN