31–40 of 77 posts
To clarify, this is SOTA in its size category, right? It's not better than Parakeet, for example?
Well, to clarify, it is both larger than parakeet in parameter count (parakeet is available in 0.6B and 1.1B), since it's 2B params, and also performs better than it on the benchmarks that hugging face publishes on the openASR leaderboard
Dumb question, but if this is "open source" is there source code somewhere? Or does that term mean something different in the world of models that must be trained to be useful?
This kids make sense because "compiling" (training) the model cost inhibitly much, and we can still benefit from the artifacts.
I can't say enough nice things about Cohere's services. I migrated over to their embedding model a few months ago for clip-style embeddings and it's been fantastic. It has the most crisp, steady P50 of any external service I've used in a long time.
can u comment on overall quality? their models tend to be a bit smaller and less performant overall.
My experience with Cohere and interacting with their sales engineers has been boring, I say that is the most flattering way possible. Embeddings are a core service at this point like VMs and DBs. They just need to work and work well and thats what they're selling.
To clarify, this is SOTA in its size category, right? It's not better than Parakeet, for example?
To clarify, this is SOTA in its size category, right? It's not better than Parakeet, for example?
Looking at the ASR leaderboard ( https://huggingface.co/spaces/hf-audio/open_asr_leaderboard ), Parakeet (.6B) is still near the top on speed, but about 10th on WER.
My worry is that ASR will end up like OCR. If the multi modal large AI system is good enough (latency wise), the advantage of domain understanding eats the other technlogies alive. In OCR, even when the characters are poorly scanned, the deep domain understanding these large multi modal AIs have allows it to understand what the document actually meant - this is going to be order id because in the million invoices I h…
Why are you 'worried' about it? Shouldn't we strive for better technology even if it means some will 'lose'?
Probably the answer is simply to tweak the metric so it's a bit more smart than WER - allow "unclear" output which is penalised less than actually incorrect answers. I'd be surprised if nobody has done that.
Multimodels are way better
So far, the best I have found while testing models for my language learning app (Copycat Cafe) is Soniox. All others performed badly for non native accents. The worst were whisper-based models because they hallucinate when they misunderstand and tend to come up with random phrases that have nothing to do with the topic.
> Limitations >Timestamps/Speaker diarization. The model does not feature either of these. What a shame. Is whisperx still the best choice if you want timestamps/diarization?
There is also: https://github.com/linto-ai/whisper-timestamped It doesn't use an extra model (so it supports every language that works with Whisper out of the box and use less memory), it works by applying Dynamic Time Warping to cross-attention weights.