Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Search inside YouTube videos using natural language queries

github.com

Post reply on HN
Print view View original

41–45 of 45 posts

Re: Show HN: Search inside YouTube videos using natural language queries

#41

Post by vladoh »

mncharity wrote: post #36

Years ago, when I prototyped an orders-of-magnitude physical-properties playground web app, I found the development bottleneck was searching for video clips and images. For example, find clips showing the heartbeat of a {goldfish,mouse,cat,dog,child,adult,horse,elephant,whale}, for a association (metabolic rate scales with body mass). Jiggling an oom kg scale, maybe you're shown a cat or mouse, and maybe a mouse hear…

I think we will get there soon! CLIP is a new model that OpenAI published in January and I'm sure Google is working on similar technology, which can be used for both video and image search.

Re: Show HN: Search inside YouTube videos using natural language queries

#42

Post by akinhwan »

vladoh wrote: post #2

This is a demonstration of the abilities of OpenAI's CLIP neural network. The tool will download a YouTube video, extract frames at regular intervals and precompute the feature vectors of each frame using CLIP. You can then use natural language search queries to find a particular frame of the video. The results are really amazing in my opinion... If you want to experiment with it yourself, I prepared a Colab notebook…

Wow what sort of business ideas do you think could come of this?

Re: Show HN: Search inside YouTube videos using natural language queries

#43

Post by gbennett71 »

hansel_der wrote: post #30
gbennett71 wrote: post #27

Downloading YouTube videos is a Terms of Use violation, no?

lol no. belive it or not, but your browser/tv has to download the video in order to show it.

https://www.youtube.com/static?template=terms&gl=AU

Permissions and Restrictions You may access and use the Service as made available to you, as long as you comply with this Agreement and applicable law. You may view or listen to Content for your personal, non-commercial use. You may also show YouTube videos through the embeddable YouTube player.

The following restrictions apply to your use of the Service. You are not allowed to:

access, reproduce, download, distribute, transmit, broadcast, display, sell, license, alter, modify or otherwise use any part of the Service or any Content except: (a) as expressly authorized by the Service; or (b) with prior written permission from YouTube and, if applicable, the respective rights holders;

access the Service using any automated means (such as robots, botnets or scrapers) except (a) in the case of public search engines, in accordance with YouTube’s robots.txt file; or (b) with YouTube’s prior written permission.

>>> So, yeah: not allowed

Re: Show HN: Search inside YouTube videos using natural language queries

#45

Post by pininja »

vladoh wrote: post #11
pininja wrote: post #5

Earlier quoted context omitted.

This is very cool! Does this produce a occurrence index by any chance? It would be neat to explore a word map of a video.

I pushed a small update and you will now see a heatmap displaying the score of the search query for each frame.

Looks great! Adds a whole dimension to the video. Thanks.
Post reply on HN