Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: A real time AI video agent with under 1 second of latency

news.ycombinator.com

Post reply on HN
Print view View original

131–140 of 264 posts

Re: Show HN: A real time AI video agent with under 1 second of latency

#131

Post by squarefoot »

aniviacat wrote: post #112
causal wrote: post #96

Earlier quoted context omitted.

This presumes the AI has the same level of problem-solving agency of a real human, which I think is really asking for AGI. Until then I expect AI chatbots will mostly succeed at portraying care and gaslighting customers without actually finding solutions.

That really depends on the type of call center we're talking about. Many (most?) call centers won't do much more than telling you to turn it off and on again, even when you're talking to a real person. (And for many cutomers, that is really all they need.)

And AI operators in those call centers wouldn't even need to be better than humans, just cheaper. Not just for saving on human hiring: no building rent, no insurance, no this and that; everything would live within a cluster somewhere.

Re: Show HN: A real time AI video agent with under 1 second of latency

#132

Post by shtack »

leobg wrote: post #102
shtack wrote: post #77

Cool, I built a prototype of something very similar (face+voice cloning, no video analysis) using openly available models/APIs: https://bslsk0.appspot.com/ The video latency is definitely the biggest hurdle. With dedicated a100s I can get it down <2s, but it's pricy.

This looks awesome. Didn’t seem to hear me, but the video looks great. Can you share what models you are using? You say these are all open models.

The model doing the heavy lifting is https://github.com/Rudrabha/Wav2Lip

Mic permissions on mobile are tricky, which might have been your issue? Note in this prototype you also need to hold the blue button down to speak.

Re: Show HN: A real time AI video agent with under 1 second of latency

#133

Post by diggan »

kabirgoel wrote: post #125
radarsat1 wrote: post #53

As someone not super familiar with deployment but enough to know that GPUs are difficult to work with due to being costly and sometimes hard to allocate: apart from optimizing the models themselves, what's the trick for handling cloud GPU resources at scale to serve something like this, supporting many realtime connections with low latency? Do you just allocate a GPU per websocket connection? Which would mean keeping…

(Not the author but I work in real-time voice.) WebSockets don't really translate to actual GPU load, since they spend a ton of time idling. So strictly speaking, you don't need a GPU per WebSocket assuming your GPU infra is sufficiently decoupled from your user-facing API code. That said, a GPU per generation (for some operational definition of "generation") isn't uncommon, but there's a standard bag of tricks, like…

> that you can use to maximize throughput

While degrading the experience sometimes, little or by a lot, thanks to possible "noisy neighbors". Worth keeping in mind that most things are trade-offs somehow :) Mostly important for "real-time" rather than batched/async stuff of course.

Re: Show HN: A real time AI video agent with under 1 second of latency

#135

Post by squarefoot »

I'm not entirely comfortable giving access to my audio/video to anyone/anything so I didn't try the demo, anyway I watched the video generation demos and they are very easily recognizable as AI, but... holy crap! Things have progressed at unbelievable speed during the last two years.

If I may offer some advice about potential uses beyond the predictable and trivial use in advertising, there's an army out there of elderly people who spend the rest of their life completely alone, either at home or hospitalized. A low cost version that worked like 1 hour a day with less aggressive reduction on latency to keep costs low could change the life of so many people.

Re: Show HN: A real time AI video agent with under 1 second of latency

#136

Post by hassaanr »

pookeh wrote: post #121

I joined while in the bathroom where the camera was facing upwards looking up to the hanging towel on the wall…and it said “looks like you got a cozy bathroom here” You have to be kidding me.

Appreciate you not flashing Carter or my digital twin haha

Re: Show HN: A real time AI video agent with under 1 second of latency

#137

Post by hyperG »

heyitsguay wrote: post #21
joshdavham wrote: post #18

Earlier quoted context omitted.

That's actually a good question. For example, the technology is still currently at a level where the user can still cleary tell that it's a chatbot, but now with a face. Does this make their experience better? Or does it add a weird level of uncaninness to the experience?

I don't think the level of fidelity actually matters as much as authority or ability. What can the agent do that isn't accomplished by, for example, a landing page or an FAQ page? I've never encountered a (text) chatbot that did anything useful for me as a consumer, whether for sales or support.

The problem is I don't even like video calls with real people.

It is the same problem that in most context, the video has no purpose. The only use for video is to put a face to a name/voice.

I hope my company competitors switch to AI video for sales and support. I would absolutely pay for that!

Re: Show HN: A real time AI video agent with under 1 second of latency

#138

Post by hassaanr »

kwindla wrote: post #74

If you're interested in low-latency, multi-modal AI, Tavus is sponsoring a hackathon Oct 19th-20th in SF. (I'm helping to organize it.) There will also be a remote track for people who aren't in SF, so feel free to sign up wherever you are in the world. https://x.com/kwindla/status/1839767364981920246

Big +1 here! Also shoutout to the Daily team who helped build this!

Re: Show HN: A real time AI video agent with under 1 second of latency

#139

Post by hassaanr »

myprotegeai wrote: post #128
kwindla wrote: post #74

If you're interested in low-latency, multi-modal AI, Tavus is sponsoring a hackathon Oct 19th-20th in SF. (I'm helping to organize it.) There will also be a remote track for people who aren't in SF, so feel free to sign up wherever you are in the world. https://x.com/kwindla/status/1839767364981920246

Can you say more about how developers will use this? Is the api going to be exposed to participants?

The API is exposed now, you can signup at tavus.io, and at the hackathon we’ll be giving credits to build!

Re: Show HN: A real time AI video agent with under 1 second of latency

#140

Post by thangalin »

jszymborski wrote: post #129

> The next worst offender was actually detecting when someone stopped speaking. ChatGPT is terrible at this in my experience. Always cuts me off.

> Always cuts me off.

In my sci-fi novel, when characters speak with their home automation system, they always have to follow the same format: "Tau, , please." It's that "please" at the end that solves the stopped speaking problem.

Am looking for alpha readers! (See profile for contact details.)

Post reply on HN