200ms Voice LLM
github.com
200ms Voice LLM
1–10 of 24 posts
Re: 200ms Voice LLM
#2I hope this catches on and can be transplanted into any LLM soon. Instant voice integration is poised to unlock many UX things like auto-pilot in cars making you a real powerful co-pilot where you can suggest fine-tune settings on the fly for the current situation or anything that changes lots of settings at once with visual feedback.
Re: 200ms Voice LLM
#3Yeah this is awesome. Keep reducing that latency, that's the path to the killer assistant.
Re: 200ms Voice LLM
#4Cannot wait for an instant translator. Something that can translate synchronously (!) one language it hears into a language that I understand. Getting closer!
Re: 200ms Voice LLM
#5Cannot wait for an instant translator. Something that can translate synchronously (!) one language it hears into a language that I understand. Getting closer!
I don't understand how that could possibly work. You will always need some time window, don't you? Also sometimes the translation of the first word can only be inferred when you process the last word in a sentence (simplified example).
Re: 200ms Voice LLM
#6Yeah this is awesome. Keep reducing that latency, that's the path to the killer assistant.
Freudian slip ?
Re: 200ms Voice LLM
#7Cannot wait for an instant translator. Something that can translate synchronously (!) one language it hears into a language that I understand. Getting closer!
I don't understand how that could possibly work. You will always need some time window, don't you? Also sometimes the translation of the first word can only be inferred when you process the last word in a sentence (simplified example).
You mean the job of a translator is by definition impossible?
Edit: Yes, for sure there will be a delay of a few words or even one or two short sentences. Just like human translators. Not a problem I think.
Edit 2: Very curious why my first comment was downvoted?
Re: 200ms Voice LLM
#8Earlier quoted context omitted.
I don't understand how that could possibly work. You will always need some time window, don't you? Also sometimes the translation of the first word can only be inferred when you process the last word in a sentence (simplified example).
You mean the job of a translator is by definition impossible? Edit: Yes, for sure there will be a delay of a few words or even one or two short sentences. Just like human translators. Not a problem I think. Edit 2: Very curious why my first comment was downvoted?
[deleted]
Re: 200ms Voice LLM
#9Earlier quoted context omitted.
I don't understand how that could possibly work. You will always need some time window, don't you? Also sometimes the translation of the first word can only be inferred when you process the last word in a sentence (simplified example).
You mean the job of a translator is by definition impossible? Edit: Yes, for sure there will be a delay of a few words or even one or two short sentences. Just like human translators. Not a problem I think. Edit 2: Very curious why my first comment was downvoted?
[deleted]
Re: 200ms Voice LLM
#10Earlier quoted context omitted.
I don't understand how that could possibly work. You will always need some time window, don't you? Also sometimes the translation of the first word can only be inferred when you process the last word in a sentence (simplified example).
You mean the job of a translator is by definition impossible? Edit: Yes, for sure there will be a delay of a few words or even one or two short sentences. Just like human translators. Not a problem I think. Edit 2: Very curious why my first comment was downvoted?
Obviously translation is possible but not "synchronously" as you wish. From what I understand 200ms is "time-to-first-token", so I still wonder how that works because, as I wrote in my comment above, typically there is no one-to-one correspondence of words/tokens from one language to another. (I didn't downvote your comment)