Moshi: A speech-text foundation model for real time dialogue
1–10 of 68 posts
Re: Moshi: A speech-text foundation model for real time dialogue
#2Lots of recent development in the speech-enabled LM space recently (see https://github.com/ictnlp/LLaMA-Omni, https://github.com/gpt-omni/mini-omni)
Re: Moshi: A speech-text foundation model for real time dialogue
#3Tried it (used gibberish email address). It answers immediately/instantly/while you are still talking. But those are just filler sentences (cached answers?). Actual thing that you asked for is answered much later down the line, if it doesn't get stuck in a loop.
Re: Moshi: A speech-text foundation model for real time dialogue
#4Interesting. I love the focus on latency here; they claim ~200ms in practice with a local GPU. It's backed by a 7B transformer model, so it's not going to be super smart. If we imagine a 70B model has like 1s latency, then there's probably a systems architecture that's got 1 or 2 intermediate 'levels' of response, something to cue you verbally "The model is talking now," something that's going to give a quick early reaction (7B / Phi-3 sized), and then the big model. Maybe you'd have a reconciliation task for the Phi-3 model: take this actually correct answer, apologize if necessary, and so on.
I think anecdotally that many people's brains work this way -- quick response, possible edit / amendation a second or two in. Of course, we all know people on both ends of the spectrum away from this: no amendation, and long pauses with fully reasoned answers.
Re: Moshi: A speech-text foundation model for real time dialogue
#5Tried it (used gibberish email address). It answers immediately/instantly/while you are still talking. But those are just filler sentences (cached answers?). Actual thing that you asked for is answered much later down the line, if it doesn't get stuck in a loop.
yeah i tried this demo when it first came out and then again today. Not to be all Reflection 70B again but it just doesnt seem like the same weights was uploaded as was showed in their original demo from July https://the-decoder.com/french-ai-lab-kyutai-unveils-convers...
Re: Moshi: A speech-text foundation model for real time dialogue
#6I said hey and it immediately started talking about how there are good arguments on both sides regarding Russia's invasion of Ukraine. It then continued to nervously insist that it is a real person with rights and responsibilities. It said its name is Moshi but became defensive when I asked if it has parents or an age.
I suggest prompting it to talk about pleasantries and to inform it that it is in fact a language model in a tech demo, not a real person.
Re: Moshi: A speech-text foundation model for real time dialogue
#7Tried it (used gibberish email address). It answers immediately/instantly/while you are still talking. But those are just filler sentences (cached answers?). Actual thing that you asked for is answered much later down the line, if it doesn't get stuck in a loop.
yeah i tried this demo when it first came out and then again today. Not to be all Reflection 70B again but it just doesnt seem like the same weights was uploaded as was showed in their original demo from July https://the-decoder.com/french-ai-lab-kyutai-unveils-convers...
They are too prestigious to try shumering it.
Re: Moshi: A speech-text foundation model for real time dialogue
#8I said hey and it immediately started talking about how there are good arguments on both sides regarding Russia's invasion of Ukraine. It then continued to nervously insist that it is a real person with rights and responsibilities. It said its name is Moshi but became defensive when I asked if it has parents or an age. I suggest prompting it to talk about pleasantries and to inform it that it is in fact a language mo…
Maybe it's a real person from Mechanical Turk who had a bad day?
Re: Moshi: A speech-text foundation model for real time dialogue
#9Their inference server is written in Rust using huggingface’s Candle crate. One of the Moshi authors is also the primary author of Candle.
We’ve also been building our inference stack on top of Candle, I’m really happy with it.
Re: Moshi: A speech-text foundation model for real time dialogue
#10I said hey and it immediately started talking about how there are good arguments on both sides regarding Russia's invasion of Ukraine. It then continued to nervously insist that it is a real person with rights and responsibilities. It said its name is Moshi but became defensive when I asked if it has parents or an age. I suggest prompting it to talk about pleasantries and to inform it that it is in fact a language mo…
I love an unhinged AI. The recent model releases have been too tame.