Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: YouTube Full Text Search – Search all of a channel from the commandline

github.com

Post reply on HN
Print view View original

51–60 of 172 posts

Re: Show HN: YouTube Full Text Search – Search all of a channel from the commandline

#51

Post by tambourine_man »

Jakob wrote: post #36
dizhn wrote: post #32

Earlier quoted context omitted.

Google doesn't even do a lot simpler things like searching by language or location. And the search is garbage. I am trying to learn Italian so that's what I am interested in but even when I enter a search term spelled correctly with its accents and everything I get anything from Brazilian Portuguese to French. They do a very helfpul translation of the term and return results that are unfortunately useless to me. (I w…

The default behaviour for multiple languages is bad, but the settings page for both region, and search results language work well. In case you haven’t seen the settings, yet.

I wish. I have set English and Portuguese only, in that order, and it keeps showing me Spanish, Italian, French.

Oh, and how about setting the default account among the many I’m logged in. Or the disable auto play setting that keeps getting reset every week.

Re: Show HN: YouTube Full Text Search – Search all of a channel from the commandline

#53

Post by belter »

derefr wrote: post #41
hysan wrote: post #12

Earlier quoted context omitted.

I would posit that google has determined that it’s more profitable to keep viewers on YouTube via controlling the viewing experience vs whatever additional ad revenue they’d gain by making videos more easily indexable. I’m basing this hypothesis on YouTube’s consistent trend of removing features related to controlling your own viewing experience. For example, removing subscription collections.

How would making videos more indexible move people off of YouTube? Once you're there, you'd stay there, with all the current recommender algorithms still in effect; all the external indexability would do is give Google (and Bing, and everyone else) more reasons to lead you into that labyrinth.

The same way they did a few weeks ago, when they changed the interface to only show you the options "Latest" and "Popular" for a channel, instead of the drop down box that was present before. You could get a list of most recent or older videos.

They are about keeping you engaged and keep suggesting to users what they think will keep you engaged with the platform. It's an entertainment platform not a library...

Re: Show HN: YouTube Full Text Search – Search all of a channel from the commandline

#54

Post by davidy123 »

gniv wrote: post #26
derefr wrote: post #3

I love that a third party is stepping up here, but it's incomprehensible to me that Google doesn't do this themselves. They're a search company, and they own YouTube. The YouTube data — including the subtitle files — is already sitting there on their servers; they don't have to scrape it, they just have to index it. What are they even doing? Fun thing to try: do a Google search with "site:youtube.com" in it. You get…

Beyond conspiracy theories it's interesting to speculate why Google is not providing native search-in-subtitles and search-in-comments. The easiest explanation is that they don't trust the quality. They probably tried it and reached the conclusion that it doesn't improve search in any meaningful direction. I know from experience that search in user reviews is very hard. Unless you really understand the review (which…

I think Google is now thoroughly infected with Big-Company-itis. A couple departments would like to and know how to comprehensively use AI across many services, which the consumers would love (though some would be confused by it). But legal, marketing, and some guy in a department called "Annex B?" are preventing it. So then the people in those departments get bored and go somewhere else and their perspectives and skills are lost.

Re: Show HN: YouTube Full Text Search – Search all of a channel from the commandline

#55

Post by cced »

ggerganov wrote: post #14

For videos without subtitles one could chain Whisper to auto-generate transcripts, though that would require downloading the audio and processing it

This is exactly what I’ve built. Nothing fancy just ytdlp + whisper + ripgrep + fzf and I’ve got a pretty interesting way to ctrl+f my YT history.

Re: Show HN: YouTube Full Text Search – Search all of a channel from the commandline

#56

Post by hnarn »

nomilk wrote: post #30

> yt-fts is a simple python script that uses yt-dlp to scrape all of a youtube channels subtitles and load them into an sqlite database that is searchable from the command line. Critically, this is per channel . I wonder if we can optionally configure this to share the downloaded transcripts to a central repository so eventually a good proportion of youtube's transcripts could be downloaded as one big text file.

> share the downloaded transcripts to a central repository

Sure, are you willing to host it and handle the absolutely inevitable legal issues?

Re: Show HN: YouTube Full Text Search – Search all of a channel from the commandline

#57

Post by cced »

liampulles wrote: post #34

Put the subs into a vector db instead and enable semantic search. :)

Can you give some additional insight as to what this enables? Maybe some additional links for research? How does one store/format data on such a database?

Re: Show HN: YouTube Full Text Search – Search all of a channel from the commandline

#58

Post by pmoriarty »

derefr wrote: post #3

I love that a third party is stepping up here, but it's incomprehensible to me that Google doesn't do this themselves. They're a search company, and they own YouTube. The YouTube data — including the subtitle files — is already sitting there on their servers; they don't have to scrape it, they just have to index it. What are they even doing? Fun thing to try: do a Google search with "site:youtube.com" in it. You get…

YouTube doesn't have to be good. It just needs to do the minimum it needs to keep users from switching to a different platform... which, because of the network effect of so many people and videos being on it, is not much at all.

If they had serious competition, they'd have to do more to keep users, but no such competition exists.

Re: Show HN: YouTube Full Text Search – Search all of a channel from the commandline

#59

Post by cced »

bigzyg33k wrote: post #37
pknerd wrote: post #28

Earlier quoted context omitted.

Pardon my ignorance as I have not worked on Vector DBs yet, could you come up with an example how it'd be different than a full text search?

Here's a (kinda) ELI5: you would use a language model to create "embeddings" of the text, which you can think of as a set of numbers representing the "meaning" of a set of characters. These numbers can be plotted as points in a space, and embeddings of things with similar meanings are plotted close to each other. So things like "exam preparation" would have embeddings close to things like "top study tips". Say you ha…

Do you have any resources that might guide one on doing something like this from scratch?

Re: Show HN: YouTube Full Text Search – Search all of a channel from the commandline

#60

Post by [deleted] »

hnarn wrote: post #56
nomilk wrote: post #30

> yt-fts is a simple python script that uses yt-dlp to scrape all of a youtube channels subtitles and load them into an sqlite database that is searchable from the command line. Critically, this is per channel . I wonder if we can optionally configure this to share the downloaded transcripts to a central repository so eventually a good proportion of youtube's transcripts could be downloaded as one big text file.

> share the downloaded transcripts to a central repository Sure, are you willing to host it and handle the absolutely inevitable legal issues?

[deleted]
Post reply on HN