Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: VideoMentions – Search YouTube based on the spoken words in videos

videomentions.com

Post reply on HN
Print view View original

81–90 of 138 posts

Re: Show HN: VideoMentions – Search YouTube based on the spoken words in videos

#81

Post by kellenmace »

mistermann wrote: post #13
kellenmace wrote: post #9

Earlier quoted context omitted.

Yeah, YouTube doesn't provide a way to search all of YouTube based on the spoken words in videos. I could update VideoMentions Search to allow users to select multiple channels, and then perform the search across all of those (maybe importing all the channels they're subscribed to could be handy... ). One way or another though, it would still require selecting specific channels to search within. That limitation notwi…

Could you do a search based on their watch history if they have it enabled?

This is a cool idea! It wouldn't apply to people who want to search all the videos on a given channel for specific keywords (including those they haven't watched). I can see it being useful for folks trying to locate a specific video they remember watching in the past, though.

One consideration is that getting the user's watch history would likely require calls to the YouTube API. So that means I would have to make this a paid service in order to offset the code of those API requests. The beautiful thing about the current iteration is that it doesn't rely on YouTube's API at all. By scraping YouTube pages and leveraging a few NPM packages, I'm currently able to offer free and unrestricted access to it.

If enough people request that ability though, I'll consider incorporating it.

Thanks for checking out my project and for the great idea!

Re: Show HN: VideoMentions – Search YouTube based on the spoken words in videos

#82

Post by kellenmace »

loxias wrote: post #78
kellenmace wrote: post #77

Earlier quoted context omitted.

Hey @loxias! Yes, you're exactly right - VideoMentions scrapes the video page markup and pulls out the "baseUrl" for the English caption track. It converts that XML caption track into JSON, then searches it for keyword matches. YouTube doesn't provide a way to search all of YouTube based on spoken words, unfortunately. Still, I think this is a useful tool for quickly locating spoken word matches within a certain chan…

> Still, I think this is a useful tool for quickly locating spoken word matches within a certain channel. Absolutely, and I agree, however,.... it necessitates that the user use or understand "channels". When I went to your site my first thought was "what the hell's a channel?" ;-) But yeah, for whatever percentage of youtube users use channels, I see this having a lot of use. My totally unsolicited advice is that yo…

"I assume you're getting hugged to death" -- lol

Yeah, maybe I'll consider having two versions of search - one version for paid customers that allows you to search for YouTube channels by name/keyword and allows you to search across all the channels you're subscribed to or across all the videos in your watch history, and a second version that's free and similar to the current iteration.

VideoMentions Search should work just fine for you - there is no server and no database behind it. When searches are performed, serverless functions are called that source data from YouTube, then return a response to the client. So it should scale just fine.

Can you please visit this link, click the button to perform a search and let me know if you see results pop up? https://videomentions.com/search?channelUrl=https%253A%252F%...

I'd love to know if you don't, or if you see any errors in the browser console. Thanks again for your thoughts on it!

Re: Show HN: VideoMentions – Search YouTube based on the spoken words in videos

#83

Post by kellenmace »

corobo wrote: post #37
kellenmace wrote: post #2

Hey HN! I just launched VideoMentions Search, a free tool that allows you to search YouTube to find videos that contain specific spoken words. Here's how to quickly try it out - Let's say you want to find every video on The Verge's YouTube channel within the last several months in which "MacBook Pro" is mentioned. Here's a pre-populated search to accomplish that: https://videomentions.com/search?channelUrl=https%253A…

Very cool! Well done! I've not got a legit use for it right now but the few example searches I thought up returned the videos I was thinking of Nice work!

Cool! Thank you!

Here are some example use-cases that I included in another comment, in case any of them are relevant to you:

1. You want to find all videos within a YouTube channel that mention your brand, your product, or the topics you care about.

2. You remember watching a video where a certain topic was discussed, and now you're trying to remember which video it was to rewatch it.

3. You run your own YouTube channel and want to quickly find the exact moments in your videos where you cover certain topics, so you can link others to that content.

Maybe #2 happens occasionally? If so, maybe you can bookmark VideoMentions Search for just such an occasion :)

In any event, thanks for checking out my project, and for the kind words!

Re: Show HN: VideoMentions – Search YouTube based on the spoken words in videos

#84

Post by kellenmace »

barefeg wrote: post #71
kellenmace wrote: post #68

Earlier quoted context omitted.

Yes, this is how VideoMentions Search works. It scrapes the video page markup and pulls out the "baseUrl" for the English caption track. It converts that XML caption track into JSON, then searches it for keyword matches. Is that what you're asking about? If you want to search within the transcript of a single video, you can accomplish that with these steps: https://kb.swtc.edu/page.php?id=90230

Ah got it. I thought there would be some API where the transcripts could be searched across videos. Maybe that requires way to many resources for Google to index

Yeah, Google is of course the king of search, so they could certainly decide to revamp YouTube search to include spoken word/transcription matches. They have all the data required to make that happen. That would make VideoMentions Search irrelevant– and I'd be okay with that!

In the meantime, I think this is a useful tool for quickly locating videos based on spoken words.

Thanks for checking out my project!

Re: Show HN: VideoMentions – Search YouTube based on the spoken words in videos

#85

Post by kellenmace »

wolongong942 wrote: post #80

Very interesting tool could see myself using it a lot. Couldn't help but search the guy that uploaded 2 million vids to yt, sorry if it breaks anything. It would be cool to search for a specific quote like "buddy of mine" (frequently said by Joe Rogan). Also searching for parts of a word (similar to regex options?) might be difficult to implement but would be super useful as an option i.e if i searched for any word t…

Hey @wolongong942! Nice- I'm glad you're finding it useful!

VideoMentions Search is built entirely using serverless technologies, so it should scale really well. There's no single server or database to act as a bottleneck. When you perform a search, your browser fires off a number of network requests to serverless function API endpoints that fetch the data from YouTube, then return a response. So if you do an "All time" search on a channel with 2 million videos, your laptop fan may kick on while your browser works hard firing off thousand and thousands of requests until it's fetched all 2 million videos, you run out of memory, or you manually hit the "Cancel" button - whichever comes first :)

Your regex idea is interesting, and I'll consider implementing some more complex rules like that if enough people request it. For now, my answer would be to either perform separate searches (like one for "yc" and another for "ycombinator"), or perform one search, then use ctrl/cmd+f to search within the matches displayed on the page.

Thanks for checking out my project, and for the great feedback!

Re: Show HN: VideoMentions – Search YouTube based on the spoken words in videos

#86

Post by kellenmace »

tomatowurst wrote: post #70
cphoover wrote: post #43

Earlier quoted context omitted.

They do... contents of speech are one of multiple input data to youtube indexing. Also you can search individual videos by showing transcript and pressing ctrl-f

I didn't know this...how do you show transcript?

Hey @tomatowurst! You can search within the spoken words/transcript of a single YouTube video by following these steps: https://kb.swtc.edu/page.php?id=90230

Re: Show HN: VideoMentions – Search YouTube based on the spoken words in videos

#88

Post by galori »

lewisjoe wrote: post #47

Hi, Great job. I was almost about to build such a thing before I realized the crazy amount of crawling and transcriptions that I have to index. How exactly did you solve/approach the problem? 1. How did you crawl across those millions of videos from the platform? 2. How are you indexing stuff like that

Its only for a single channel. That's the answer, it queries in real time.

Re: Show HN: VideoMentions – Search YouTube based on the spoken words in videos

#89

Post by kellenmace »

HidyBush wrote: post #31

Are we sure YouTube doesn't already do this? Often I have searched for a video while only remembering a certain phrase or even a certain comment left under it, and YouTube was able to actually find it

Hey @HidyBush! No, YouTube does not reliably include spoken word/transcript matches in search results. I've run tests where I open up the transcript of a video using these steps (https://kb.swtc.edu/page.php?id=90230), copy a few of the words, then perform a search using those words across that channel, and the video I copied them from doesn't appear on the list of results.

That's why VideoMentions Search exists. It provides a way to search the videos within a YouTube channel to reliably find spoken word matches (and it includes matches in the title & description, too).

Thanks for checking out my project! Please let me know if I can answer any questions about it.

Re: Show HN: VideoMentions – Search YouTube based on the spoken words in videos

#90

Post by kellenmace »

cphoover wrote: post #43
HidyBush wrote: post #31

Are we sure YouTube doesn't already do this? Often I have searched for a video while only remembering a certain phrase or even a certain comment left under it, and YouTube was able to actually find it

They do... contents of speech are one of multiple input data to youtube indexing. Also you can search individual videos by showing transcript and pressing ctrl-f

I have come to the opposite conclusion - YouTube does not reliably include spoken word/transcript matches in search results.

As I wrote in another comment, I've run tests where I open up the transcript of a video using these steps (https://kb.swtc.edu/page.php?id=90230), copy a few of the words, then perform a search using those words across that channel, and the video I copied the words from doesn't appear on the list of results.

If Google decides that it's going to revamp YouTube search to include spoken word/transcription matches, that would make VideoMentions Search irrelevant– and I'd be okay with that!

In the meantime, I think this is a useful free tool for quickly finding spoken word matches within specific channels.

Thanks for checking it out!

Post reply on HN