Show HN: Port of OpenAI's Whisper model in C/C++ • Hacker News Board

Re: Show HN: Port of OpenAI's Whisper model in C/C++

#91

Post by NWoodsman » Sat, Dec 10, 2022, 10:26 PM UTC

stevenhuang wrote: post #46

AMICABoard wrote: post #32

I vouch for this. Pretty solid and keeps improving. The OP is in the class of Magic Wizards of programming like Fabrice Bellard! There are frequent updates and performance improvements. There is also a small community of active users around this. All most all feedbacks get implemented and the OP is very responsive. The OP made it possible to do state of the art voice recognition without the PyTorch baggage and in C/C…

Yup this is so magical. I've always felt there was something off about requiring setting up what is essentially a pytorch/ml dev environment everytime end users "just" want to run inference. A single binary that does this all w/o the python stack is just incredible! edit: Got it going in 1 min! I grabbed the prebuilt artifacts (windows) - https://github.com/ggerganov/whisper.cpp/actions/runs/363552... Then downloaded…

Ok nevermind, figured it out, it requires login. Then the archive is at the very bottom of the page.

Re: Show HN: Port of OpenAI's Whisper model in C/C++

#92

Post by NWoodsman » Sat, Dec 10, 2022, 10:44 PM UTC

For those who want to try, here are the steps I took over about an hour to set it up:

1. Downloaded the Win10 artifact: https://github.com/ggerganov/whisper.cpp/actions/runs/363552... at the bottom of the page by logging in to Github. Extract and placed this folder in my F:\ drive renaming it to 'Whisper'.

2. Downloaded `ggml-large.bin` here: https://huggingface.co/datasets/ggerganov/whisper.cpp/tree/m.... Within F:\Whisper, add a folder named 'models'. Move ggml-large.bin to the 'models' folder.

3. Downloaded ffmpeg, extracted the archive to F:\FFMpeg, and set the environment variable by going to (right click) This PC -> Properties -> Advanced system settings -> (Advanced tab) -> Environment Variables -> click Path -> Edit -> (paste in ffmpegs path i.e. F:\FFMpeg\)

4. Use PowerShell to run ffmpeg against an mp3 file, to convert it to WAV (which is the only format that works) i.e.:

ffmpeg -i F:\Rec\input.mp3 -ar 16000 -ac 1 -c:a pcm_s16le F:\Output\output.wav

5. Open PowerShell again, `cd` to the Whisper folder, and ran this:

./main -m models/ggml-large.bin -f F:\Rec\output.wav

Re: Show HN: Port of OpenAI's Whisper model in C/C++

#93

Post by ggerganov » Wed, Dec 14, 2022, 8:39 AM UTC

wfcollins wrote: post #36

Hi Georgi, I am experimenting with your code now. Is there a way to force Whisper to only consider a limited vocabulary and then respond with confidence levels? I am working on an app where it is important to restrict answers and I would like to know how confident that a response is one of a set of words. If the answer could be word A with a confidence level of 95% and word B with a level of 50%, I would want to know…

A follow up on this - I came up with an interesting strategy to achieve this. Still a prototype, but I think it looks very promising:

https://github.com/ggerganov/whisper.cpp/pull/271

The source code is in the `command.cpp` and I will soon write some more details how it works. If you give it a try, definitely let me know if it worked for you.