High-fidelity simultaneous speech-to-speech translation

#1

Post by Bluestein » Thu, Jul 03, 2025, 8:27 PM UTC

High-fidelity simultaneous speech-to-speech translation
arxiv.org

Re: High-fidelity simultaneous speech-to-speech translation

#2

Post by benlivengood » Thu, Jul 03, 2025, 9:09 PM UTC

Now to get the model to run in an earbud...

Re: High-fidelity simultaneous speech-to-speech translation

#3

Post by lapink » Thu, Jul 03, 2025, 9:18 PM UTC

benlivengood wrote: post #2

Now to get the model to run in an earbud...

The model can actually run on an iPhone 16 Pro, so if the earbud is connected to one that could work!

Re: High-fidelity simultaneous speech-to-speech translation

#4

Post by wedn3sday » Thu, Jul 03, 2025, 9:28 PM UTC

For anyone else looking for examples: https://huggingface.co/spaces/kyutai/hibiki-samples

Re: High-fidelity simultaneous speech-to-speech translation

#5

Post by AIorNot » Thu, Jul 03, 2025, 9:44 PM UTC

this is amazing - love to play with this- what about other languages besides french to english

Re: High-fidelity simultaneous speech-to-speech translation

#6

Post by lapink » Thu, Jul 03, 2025, 9:54 PM UTC

AIorNot wrote: post #5

this is amazing - love to play with this- what about other languages besides french to english

Adding more languages is definitely planned! This was Tom (the first author) master’s internship project with Kyutai, and it was easier to prototype the idea with a single pair. Also he will be presenting this work at ICML in two weeks if anyone is around and wants to learn more.

Re: High-fidelity simultaneous speech-to-speech translation

#7

Post by Bluestein » Thu, Jul 03, 2025, 9:57 PM UTC

lapink wrote: post #3

benlivengood wrote: post #2

Now to get the model to run in an earbud...

The model can actually run on an iPhone 16 Pro, so if the earbud is connected to one that could work!

That would be insane.-

Thinking of it, the whole "stack" from earbuds to phone to cloud - even in just something so "commonplace" as Assistant or Alexa ...

... Is amazing: All that computing power at our disposal.-

Re: High-fidelity simultaneous speech-to-speech translation

#8

Post by iambateman » Thu, Jul 03, 2025, 10:03 PM UTC

This is why I wonder about the value of language learning for reasons other than “I’m really passionate about it.”

We are so close to interfaces that reduce the language barrier by a lot…

Re: High-fidelity simultaneous speech-to-speech translation

#9

Post by cs702 » Thu, Jul 03, 2025, 10:05 PM UTC

Nice. I'm impressed.

Translator jobs are going to go poof! overnight.

Just sayin'.

Re: High-fidelity simultaneous speech-to-speech translation

#10

Post by gagabity » Thu, Jul 03, 2025, 10:19 PM UTC

Yandex Browser has been doing this for Russian for a while, if you go to YT it offers to translate to Russian, it does multiple speakers and voices from what I remember. Not sure if all the technicalities are the same.