21–30 of 60 posts
The only thing Whisper misses is speaker diarization. I'm currently working on a model that uses Whisper + pyannote to transcribe Interviews and also detects who is speaking. It's working but damn it takes so long
I'm badly looking for that! Is there a repo I can follow?
Earlier quoted context omitted.
I'm badly looking for that! Is there a repo I can follow?
I can share my repo when it's finished. In the meantime, you can take a look at this: https://huggingface.co/spaces/vumichien/whisper-speaker-diar...
Earlier quoted context omitted.
I can share my repo when it's finished. In the meantime, you can take a look at this: https://huggingface.co/spaces/vumichien/whisper-speaker-diar...
My goal for my project is to build a tool that transcribes Interviews (e.g, in Sales or Recruiting) and puts the Transcription through ChatGPT (Waiting for the API atm) to make a summary that looks like the notes of the call. Speaker diarization is important, so I don't have more than 4000 tokens input in ChatGPT. I will see how it goes, but if it's reliable enough (looks like it so far), it will save the time it tak…
Edit: Looked at your link and I misunderstood. I think I understand you're waiting for the ChatGPT specific model now?
Earlier quoted context omitted.
My goal for my project is to build a tool that transcribes Interviews (e.g, in Sales or Recruiting) and puts the Transcription through ChatGPT (Waiting for the API atm) to make a summary that looks like the notes of the call. Speaker diarization is important, so I don't have more than 4000 tokens input in ChatGPT. I will see how it goes, but if it's reliable enough (looks like it so far), it will save the time it tak…
Why are you waiting for the API? The OpenAI Playground has API examples you can copy paste. You can go over 4000 tokens if you have a business justification and payment method. You have access to most of their models even the new Codex ones Edit: Looked at your link and I misunderstood. I think I understand you're waiting for the ChatGPT specific model now?
That's incorrect
By the way there is also another project called Whisper.cpp: https://github.com/ggerganov/whisper.cpp Which uses x8 less memory than the Python implementation for the tiny model. It would be a good idea to keep an eye on it since there are Python bindings planned on the roadmap: https://github.com/ggerganov/whisper.cpp#bindings
https://github.com/kardianos/audioclerk
Built in Go/cgo.
Is it possible to create a streaming endpoint that returns real-time transcriptions?
You could do better by overlapping the segments, except then stitching the transcriptions together becomes an issue since whisper doesn't provide reliable per-token timestamps [0], and the output of the common part of overlapping segments isn't necessarily the same. I can imagine a cool approach where you transcribe long, overlapping chunks in real-time and intelligently merge the stream of words somehow though.
Some more useful discussion here (whisper.cpp project, but still relevant) [1].