Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: LemonSlice – Upgrade your voice agents to real-time video

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 135 posts

Re: Show HN: LemonSlice – Upgrade your voice agents to real-time video

#51

Post by pickleballcourt »

One thing I've learnt from movie production is actually what separates professional from amateur quality is in the audio itself. Have you thought about implementing personaplex from NVDIA or other voice models that can both talk and listen at the same time?

Currently the conversation still feels too STT-LLM-TTS that I think a lot of the voice agents suffer from (Seems like only Sesame and NVDIA so far have nailed the natural conversation flow). Still, crazy good work train your own diffusion models, I remember taking a look at the latest literature on diffusion and was mind blown by the advances in last years or so since u-net architecture days.

EDIT: I see that the primary focus is on video generation not audio.

Re: Show HN: LemonSlice – Upgrade your voice agents to real-time video

#53

Post by lcolucci »

pickleballcourt wrote: post #51

One thing I've learnt from movie production is actually what separates professional from amateur quality is in the audio itself. Have you thought about implementing personaplex from NVDIA or other voice models that can both talk and listen at the same time? Currently the conversation still feels too STT-LLM-TTS that I think a lot of the voice agents suffer from (Seems like only Sesame and NVDIA so far have nailed the…

This is a good point on audio. Our main priority so far has been reducing latency. In service of that, we were deep in the process of integrating Hume's two-way S2S voice model instead of ElevenLabs. But then we realized that ElevenLabs had made their STT-LLM-TTS pipeline way faster in the past month and left it at that. See our measurements here (they're super interesting): https://docs.google.com/presentation/d/18kq2JKAsSahJ6yn5IJ9g...

But, to your point, there are many benefits of two-way S2S voice beyond just speed.

Using our LiveKit integration you can use LemonSlice with any voice provider you like. The current S2S providers LiveKit offers include OpenAI, Gemini, and Grok and I'm sure they'll add Personaplex soon.

Re: Show HN: LemonSlice – Upgrade your voice agents to real-time video

#55

Post by tmshapland »

lcolucci wrote: post #16
jfaat wrote: post #10

Earlier quoted context omitted.

I'm using an openAI realtime voice with livekit, and they said they have a livekit integration so it would probably be doable that way. I haven't used video in livekit though and I don't know how the plugins are setup for it

Yes this is exactly right. Using the LiveKit integration you can add LemonSlice as an avatar layer on top of any voice provider

Here's the link to the LiveKit LemonSlice plugin. It's very easy to get started. https://docs.livekit.io/agents/models/avatar/plugins/lemonsl...

Re: Show HN: LemonSlice – Upgrade your voice agents to real-time video

#57

Post by dang »

ProjectBarks wrote: post #56

Removing - Realized I made a mistake

I don't see any evidence that r0fl's comments are astroturfing. Sometimes people are just enthusiastic.

I appreciate your concern for the quality of the site - that fact that the community here cares so much about protecting it is the main reason why it continues to survive. Still, it's against HN's rules to post like you did here. Could you please review https://news.ycombinator.com/newsguidelines.html? Note this part:

"Please don't post insinuations about astroturfing, shilling, bots, brigading, foreign agents and the like. It degrades discussion and is usually mistaken. If you're worried about abuse, email hn@ycombinator.com and we'll look at the data."

Re: Show HN: LemonSlice – Upgrade your voice agents to real-time video

#58

Post by sid-the-kid »

ProjectBarks wrote: post #56

Removing - Realized I made a mistake

it's a fair concern. but, we don't know r0fl. and we are not astroturfing.

even I am surprised with how many opnely positive comments we are getting. it's not been our experience in the past.

Re: Show HN: LemonSlice – Upgrade your voice agents to real-time video

#59

Post by jedwhite »

That's an interesting insight about "stacking tricks" together. I'm curious where you found that approach hit limits. And what gives you an advantage if anything against others copying it. Getting real-time streaming with a 20B parameter diffusion model and 20fps on a single GPU seems objectively impressive. It's hard to resist just saying "wow" looking at the demo, but I know that's not helpful here. It is clearly a substantial technical achievement and I'm sure lots of other folks here would be interested in the limits with the approach and how generalizable it is.

Re: Show HN: LemonSlice – Upgrade your voice agents to real-time video

#60

Post by pickleballcourt »

lcolucci wrote: post #53
pickleballcourt wrote: post #51

One thing I've learnt from movie production is actually what separates professional from amateur quality is in the audio itself. Have you thought about implementing personaplex from NVDIA or other voice models that can both talk and listen at the same time? Currently the conversation still feels too STT-LLM-TTS that I think a lot of the voice agents suffer from (Seems like only Sesame and NVDIA so far have nailed the…

This is a good point on audio. Our main priority so far has been reducing latency. In service of that, we were deep in the process of integrating Hume's two-way S2S voice model instead of ElevenLabs. But then we realized that ElevenLabs had made their STT-LLM-TTS pipeline way faster in the past month and left it at that. See our measurements here (they're super interesting): https://docs.google.com/presentation/d/18k…

Thanks for sharing! Makes sense to go with latency first.
Post reply on HN