Serverless Video Transcription inspired by Cyberpunk 2077

#1

Post by pierremenard » Thu, Dec 22, 2022, 10:20 PM UTC

Serverless Video Transcription inspired by Cyberpunk 2077
github.com

Re: Serverless Video Transcription inspired by Cyberpunk 2077

#2

Post by chinchilla2020 » Thu, Dec 22, 2022, 11:16 PM UTC

This is impressive and so is the writeup. How did you create the diagrams in the readme? I really like that visual format for diagrams.

Re: Serverless Video Transcription inspired by Cyberpunk 2077

#3

Post by navanchauhan » Thu, Dec 22, 2022, 11:16 PM UTC

Pretty darn cool!

I almost want to try and implement this in OpenCVjs and see if it is possible to build a browser extension to do this in the browser itself

Re: Serverless Video Transcription inspired by Cyberpunk 2077

#4

Post by super_linear » Thu, Dec 22, 2022, 11:21 PM UTC

chinchilla2020 wrote: post #2

This is impressive and so is the writeup. How did you create the diagrams in the readme? I really like that visual format for diagrams.

For the hand-drawn style ones, I think it's https://excalidraw.com/

Re: Serverless Video Transcription inspired by Cyberpunk 2077

#5

Post by ArekDymalski » Thu, Dec 22, 2022, 11:21 PM UTC

Now this is both cool piece of tech and practically useful. Very nice!

Re: Serverless Video Transcription inspired by Cyberpunk 2077

#6

Post by vorpalhex » Thu, Dec 22, 2022, 11:25 PM UTC

This would be a killer app for a google glass type device. A near realtime closed captioning device with translation.

Re: Serverless Video Transcription inspired by Cyberpunk 2077

#7

Post by totetsu » Thu, Dec 22, 2022, 11:54 PM UTC

>Matching faces to voices relies on simple co-occurence heuristic, and will not work in certain scenarios (e.g. if the whole conversation between two people is recorded from a single angle)

This seems like the really hard part. maybe if there was a way to find the time lips moves for a face. or guess gender and age of both face and voice.. or If the audio is a stereo mix, using relative position

Re: Serverless Video Transcription inspired by Cyberpunk 2077

#8

Post by photoGrant » Fri, Dec 23, 2022, 12:24 AM UTC

totetsu wrote: post #7

>Matching faces to voices relies on simple co-occurence heuristic, and will not work in certain scenarios (e.g. if the whole conversation between two people is recorded from a single angle) This seems like the really hard part. maybe if there was a way to find the time lips moves for a face. or guess gender and age of both face and voice.. or If the audio is a stereo mix, using relative position

There's this that can differentiate speakers

https://github.com/CorentinJ/Real-Time-Voice-Cloning

Re: Serverless Video Transcription inspired by Cyberpunk 2077

#9

Post by holler » Fri, Dec 23, 2022, 12:25 AM UTC

This is awesome, impressed you threw this together over a weekend!

What did you use to make that entity diagram?

edit: answered below

Re: Serverless Video Transcription inspired by Cyberpunk 2077

#10

Post by abidlabs » Fri, Dec 23, 2022, 12:30 AM UTC

Neat use of Gradio & Modal