481–490 of 508 posts
A second run gave better results, but in most runs I do see instances where phrases repeat from 2-20 times.
Earlier quoted context omitted.
I suspect this is coming. I mean we do have decent text to speech systems already, but in this vein of “we used neural networks and now it’s very very good” you can imagine that with something like GPT-3, to extend it they could use this speech to text system so you could speak to it for input, and then a natural progression is that it can use text to speech to return the output, so you just have a voice oriented con…
A full NLP system would include speech recognition, TTS, a large language model, and a vector search engine. The LM should be multi modal, multi language and multi task, "multi-multi-model" for short haha. I'm wondering when we'll have this stack as default on all OSes. We want to be able to search, transcribe, generate speech, run NLP tasks on the language model and integrate with external APIs by intent detection.…
But yeah you make a good point about LLMs being too large to run on a normal PC. I do somewhat suspect that we might see some rapid acceleration in the size of neural network processors as large models begin to offer more utility. I think for now they have limited appeal but we’re already seeing things like Tesla’s Dojo make large leaps in capability to rapidly process complex networks.
In five to ten years we may see built in accelerators come standard in most computers capable of running very complex models. Already Apple provides ever more powerful accelerators in their phones. You could imagine Adobe offering real time diffusion models as part of Photoshop, among other things.
Earlier quoted context omitted.
This is only one side of the coin, you still need really good models for Speech Synthesis and then be able to have it all working in almost real time, ideally locally on device.
As far as TTS goes, Mycroft.ai[0] has released a decent offline one. [0] https://mycroft.ai/
I was comparing a batch of transcriptions between these models and vosk, and noticed that the medium.en model produces some weird results compared to the others. I've seen a number of loops with one word or a small sequence of words repeating several times. It seems more prone to output that reads like nonsense than the others. More troubling is a short audio clip that got a few full sentences back, several times the…
The whole value of this model is in 680 000 hours of training data and to reuse this value you need large model, not smaller ones. Smaller versions just don't have enough capacity to represent training data properly.
It's the only one that seems to be occasionally spitting out significant chunks of training data versus something that resembles the audio.
This is an astonishing package. Every AI voice-to-text model I've tried on "The Wire's" famous "fuck" scene [0] usually fails, because the youtube clip's audio quality is bad and it's a scene with virtually no dialogue except breathing and "Fuck". But Whisper returned impressive results [1] [0] https://www.youtube.com/watch?v=DS6pE88Xg3s [1] $ yt-dlp --extract-audio --audio-format mp3 -o wire-fuck.mp3 https://www.you…
Earlier quoted context omitted.
It was a credible argument when GPT-3 was released. But now there are open models that are as capable as GPT-3 and that mayhem has not materialized, with the possible exception of GPT-4chan. They could release it now under a non-commercial license, if they cared to.
Can you provide an example of an open model as capable as GPT-3? I know there's some "mini-GPT" type models around, but they don't seem nearly as capable.
As mentioned elsewhere in the thread there are some large models around the 50-200B band that compete directly with GPT-3, but I haven’t used these.
Be wary of using this model - the licensing of this model seems sketchy. Several of the datasets used for training like WSJ and TED-LIUM have clear non-commercial clauses. I'm not a lawyer but releasing a model as "MIT" seems dubious, and hopefully OpenAI has paid for the appropriate licenses during training as they are no longer a research-only non profit.
Any opinions on what this means for speech-to-text companies like rev.ai and assmembly.ai ? We've tested open source solutions for s2t, like kaldi, but the quality was not good enough. However, one of the main advantages of a service like assembly.ai to me was that they offer sentence splitting in form of punctuation and speaker detection, which Kaldi does not. So I guess I answered my own question to some degree: A…
You can apply public punctation model from Vosk on top of Kaldi output, you can also get speaker labels with existing open source software. On quick video transcription test this model is more accurate than AssemblyAI and Rev AI. It will be harder for them to sell pure ASR now. Some more business-oriented applications will still be important though, for example ASR as part of callcenter analytics solution or as a par…
Hello Nickolay :)
Diarization has always been the hard part for me, especially since it is very difficult to do comparisons within your domain. The evaluation metrics are not descriptive enough imo.
Would you say Titanet or EcapaTDNN are decent for use in production alongside, say, Whisper, or any other ASR output, if given the timestamps, so as to bypass running VAD? I'm just about to run experiments to try pyannote's diarization model and google's uis-rnn to test out how well they work, but it's a tad beyond my ability to evaluate.
I also wonder if Whisper architecture would be good for generating embeddings, but I feel it's focused so much on what is said rather than how it's said that it might not transfer over well to speaker tasks.
Earlier quoted context omitted.
I suspect Whisper is more robust than other "SOTA" models, but this release is likely leaving a fair bit of accuracy on the table considering the amount of resources OpenAI is capable of throwing at training it. Comparing the readily available test sets from the paper to some of my personal robust models (for the Talon models, this is greedy decoding, no language model): Talon Talon Talon Whisper wav2vec 2.0 28M 300M…
It is interesting how they compare with wav2vec2 instead of nemo conformer (which is more accurate) in Table 2.
On that note, a core Nvidia NeMo developer I follow posted this: https://twitter.com/HaseoX94/status/1572748653189791745
He calls it a "T5 for ASR" paper :) More insights in there, have a look! Curious to see what your blog would put up as well!
Earlier quoted context omitted.
Tried again with Blaise Pascal -- the famous fragment of a letter where he says he's sorry he didn't have enough time to make it shorter. Original: > Mes révérends pères, mes lettres n’avaient pas accoutumé de se suivre de si près, ni d’être si étendues. Le peu de temps que j’ai eu a été cause de l’un et de l’autre. Je n’ai fait celle-ci plus longue que parce que je n’ai pas eu le loisir de la faire plus courte. La r…
Depends on the way you're pronouncing it maybe. To be intelligible IMO it must be read differently from a modern text, with well sounding liaisons, and all vowels very distinct: "un" sounds differently from "in", "â" clearly differs from "a", "ai" and "è" from "é" and for instance the "e" in "étendues" must be pronounced, though not loudly. My test gives that, much better than yours: Mes *rêverants* pères, mes lettre…