Whisper Jax: Transcribe a 1 hour of audio in under 15 seconds
1–10 of 11 posts
Re: Whisper Jax: Transcribe a 1 hour of audio in under 15 seconds
#2Whisper JAX is an optimised implementation of the Whisper model by OpenAI. It runs on JAX with a TPU v4-8 in the backend. Compared to PyTorch on an A100 GPU, it is over 70x faster, making it the fastest Whisper API available.
Re: Whisper Jax: Transcribe a 1 hour of audio in under 15 seconds
#3[deleted]
Re: Whisper Jax: Transcribe a 1 hour of audio in under 15 seconds
#4Whisper JAX is an optimised implementation of the Whisper model by OpenAI. It runs on JAX with a TPU v4-8 in the backend. Compared to PyTorch on an A100 GPU, it is over 70x faster, making it the fastest Whisper API available.
how does it compare to whisper.cpp?
Re: Whisper Jax: Transcribe a 1 hour of audio in under 15 seconds
#5Re: Whisper Jax: Transcribe a 1 hour of audio in under 15 seconds
#6I've been looking for faster implementations of Whisper, the main drawback with Whisper Jax is that the performance comes from running on Google TPUs, which are much more expensive than GPUs.
On "normal" GPUs the fastest implementation I've found is https://github.com/guillaumekln/faster-whisper. Whisper.cpp works faster on a CPU, especially on Apple Silicon, but still nowhere near the performance you could get on a GPU (understandably).
How does Whisper Jax compares to faster-whisper on a GPU?
Re: Whisper Jax: Transcribe a 1 hour of audio in under 15 seconds
#7I've been looking for faster implementations of Whisper, the main drawback with Whisper Jax is that the performance comes from running on Google TPUs, which are much more expensive than GPUs. On "normal" GPUs the fastest implementation I've found is https://github.com/guillaumekln/faster-whisper . Whisper.cpp works faster on a CPU, especially on Apple Silicon, but still nowhere near the performance you could get on a…
Yea, whisper runs fairly quickly on really old GPUs and leveraging CTranslate2 (underlying tech in your link) you need This might be faster but an order of magnitude more expensive? Also, huge lockin to one provider.
Re: Whisper Jax: Transcribe a 1 hour of audio in under 15 seconds
#8Whisper JAX is an optimised implementation of the Whisper model by OpenAI. It runs on JAX with a TPU v4-8 in the backend. Compared to PyTorch on an A100 GPU, it is over 70x faster, making it the fastest Whisper API available.
how does it compare to whisper.cpp?
Not a good comparison. CPU vs TPU. Also, whisper.cpp perf varies a ton depending on hardware. Avx512 or neural engine?
Re: Whisper Jax: Transcribe a 1 hour of audio in under 15 seconds
#9I've been looking for faster implementations of Whisper, the main drawback with Whisper Jax is that the performance comes from running on Google TPUs, which are much more expensive than GPUs. On "normal" GPUs the fastest implementation I've found is https://github.com/guillaumekln/faster-whisper . Whisper.cpp works faster on a CPU, especially on Apple Silicon, but still nowhere near the performance you could get on a…
jax and whisper-jax support running on GPUs in addition to TPUs, this particular demo just runs on a TPU.
Re: Whisper Jax: Transcribe a 1 hour of audio in under 15 seconds
#10Could this be used on device to generate closed captions of video on the fly?