Show HN: Search inside YouTube videos using natural language queries
1–10 of 45 posts
Re: Show HN: Search inside YouTube videos using natural language queries
#2This is a demonstration of the abilities of OpenAI's CLIP neural network. The tool will download a YouTube video, extract frames at regular intervals and precompute the feature vectors of each frame using CLIP.
You can then use natural language search queries to find a particular frame of the video. The results are really amazing in my opinion...
If you want to experiment with it yourself, I prepared a Colab notebook that can easily be run: https://colab.research.google.com/github/haltakov/natural-la...
Re: Show HN: Search inside YouTube videos using natural language queries
#3notebook isn't loading for me but this seems really cool
Re: Show HN: Search inside YouTube videos using natural language queries
#4notebook isn't loading for me but this seems really cool
Hmm, this is strange... The Colab notebook should load even if you are not logged in with your Google account (you will need to log in if you want to run it, though).
Re: Show HN: Search inside YouTube videos using natural language queries
#5This is a demonstration of the abilities of OpenAI's CLIP neural network. The tool will download a YouTube video, extract frames at regular intervals and precompute the feature vectors of each frame using CLIP. You can then use natural language search queries to find a particular frame of the video. The results are really amazing in my opinion... If you want to experiment with it yourself, I prepared a Colab notebook…
This is very cool! Does this produce a occurrence index by any chance? It would be neat to explore a word map of a video.
Re: Show HN: Search inside YouTube videos using natural language queries
#6This is a demonstration of the abilities of OpenAI's CLIP neural network. The tool will download a YouTube video, extract frames at regular intervals and precompute the feature vectors of each frame using CLIP. You can then use natural language search queries to find a particular frame of the video. The results are really amazing in my opinion... If you want to experiment with it yourself, I prepared a Colab notebook…
This is very cool! Does this produce a occurrence index by any chance? It would be neat to explore a word map of a video.
Not yet, but I had this idea as well. You basically get a score describing how well a phrase is matching each of the images so it won't be difficult to do. I'll look into that!
Re: Show HN: Search inside YouTube videos using natural language queries
#7This is a demonstration of the abilities of OpenAI's CLIP neural network. The tool will download a YouTube video, extract frames at regular intervals and precompute the feature vectors of each frame using CLIP. You can then use natural language search queries to find a particular frame of the video. The results are really amazing in my opinion... If you want to experiment with it yourself, I prepared a Colab notebook…
Just a heads up: in the demo's setup block, it installs pytorch 1.7.0 + cu101 from CLIP (since it requires them) and then immediately uninstalls it to re-install pytorch 1.7.1 by the next command, which takes at least 5 minutes. If we don't really need 1.7.1, we can save some time by removing the manual pytorch installation line.
Re: Show HN: Search inside YouTube videos using natural language queries
#8This is a demonstration of the abilities of OpenAI's CLIP neural network. The tool will download a YouTube video, extract frames at regular intervals and precompute the feature vectors of each frame using CLIP. You can then use natural language search queries to find a particular frame of the video. The results are really amazing in my opinion... If you want to experiment with it yourself, I prepared a Colab notebook…
Just a heads up: in the demo's setup block, it installs pytorch 1.7.0 + cu101 from CLIP (since it requires them) and then immediately uninstalls it to re-install pytorch 1.7.1 by the next command, which takes at least 5 minutes. If we don't really need 1.7.1, we can save some time by removing the manual pytorch installation line.
Yes, I know that this is a bit slow. The problem is you really need 1.7.1, because 1.7.0 leads to some strange issues and broken results:
https://github.com/openai/CLIP/issues/13#issuecomment-771143...
Re: Show HN: Search inside YouTube videos using natural language queries
#9Earlier quoted context omitted.
Just a heads up: in the demo's setup block, it installs pytorch 1.7.0 + cu101 from CLIP (since it requires them) and then immediately uninstalls it to re-install pytorch 1.7.1 by the next command, which takes at least 5 minutes. If we don't really need 1.7.1, we can save some time by removing the manual pytorch installation line.
Yes, I know that this is a bit slow. The problem is you really need 1.7.1, because 1.7.0 leads to some strange issues and broken results: https://github.com/openai/CLIP/issues/13#issuecomment-771143...
Ah, got it. I just noticed 1.7.1 is already in CLIP's requirements.txt, weird that colab would still install 1.7.0 to begin with.
Edit: I just realized 1.7.0 just came with colab, not installed by CLIP.
Re: Show HN: Search inside YouTube videos using natural language queries
#10Earlier quoted context omitted.
Yes, I know that this is a bit slow. The problem is you really need 1.7.1, because 1.7.0 leads to some strange issues and broken results: https://github.com/openai/CLIP/issues/13#issuecomment-771143...
Ah, got it. I just noticed 1.7.1 is already in CLIP's requirements.txt, weird that colab would still install 1.7.0 to begin with. Edit: I just realized 1.7.0 just came with colab, not installed by CLIP.
Yes, this is the actual problem...