Lex Fridman Podcast search engine

#1

Post by joemckenna » Thu, Dec 08, 2022, 11:26 PM UTC

Lex Fridman Podcast search engine
lexpodsearch.com

Re: Lex Fridman Podcast search engine

#2

Post by joemckenna » Thu, Dec 08, 2022, 11:26 PM UTC

As a side project, I created a search engine for the Lex Fridman podcast. It uses https://huggingface.co/sentence-transformers/all-MiniLM-L6-v... to encode the YouTube auto-generated captions. Feedback is greatly appreciated!

Re: Lex Fridman Podcast search engine

#3

Post by 1970-01-01 » Thu, Dec 08, 2022, 11:57 PM UTC

Roger Penrose is missing from it?

Re: Lex Fridman Podcast search engine

#4

Post by joemckenna » Fri, Dec 09, 2022, 12:10 AM UTC

1970-01-01 wrote: post #3

Roger Penrose is missing from it?

For some videos, embedding on third-party sites has been disabled. Previously filtered, but revised now. Thanks!

Re: Lex Fridman Podcast search engine

#5

Post by gbertb » Fri, Dec 09, 2022, 1:34 AM UTC

This is awesome! Do you plan on open sourcing the code by any chance?

Re: Lex Fridman Podcast search engine

#6

Post by joemckenna » Fri, Dec 09, 2022, 2:13 AM UTC

gbertb wrote: post #5

This is awesome! Do you plan on open sourcing the code by any chance?

Sure! https://github.com/joepatmckenna/podsearch. Thanks!

Re: Lex Fridman Podcast search engine

#7

Post by gbertb » Fri, Dec 09, 2022, 5:15 AM UTC

joemckenna wrote: post #6

gbertb wrote: post #5

This is awesome! Do you plan on open sourcing the code by any chance?

Sure! https://github.com/joepatmckenna/podsearch . Thanks!

thank you!

Re: Lex Fridman Podcast search engine

#8

Post by pieterr » Fri, Dec 09, 2022, 6:23 AM UTC

'rossum' doesn't return any results?

Re: Lex Fridman Podcast search engine

#9

Post by joemckenna » Sat, Dec 10, 2022, 5:11 AM UTC

pieterr wrote: post #8

'rossum' doesn't return any results?

I was only matching queries against parts of the YouTube auto-generated captions. In theory, this should work for queries that are guests' names because Lex states the guest name in the beginning of the episode. But, in practice it doesn't because of transcription errors and how results are aggregated/sorted/filtered. Now, I'm matching queries against text that accompanies each episode as well as the transcripts and separating results into "Episodes" and "Clips" sections. This seems to enable querying by guest. Thanks for the feedback!