Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Real-time AI Voice Chat at ~500ms Latency

github.com

Post reply on HN
Print view View original

191–200 of 238 posts

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#191

Post by genewitch »

mvdtnz wrote: post #31
WhitneyLand wrote: post #16

Earlier quoted context omitted.

>>they don't allow talking naturally Neither do phone calls. Round trip latency can easily be 300ms, which we’ve all learned to adapt our speech to. If you want to feel true luxury find an old analog PTSN line. No compression artifacts or delays. Beautiful and seamless 50ms latency. Digital was a terrible event for call quality.

I don't know how your post is relevant to the discussion of AI models interrupting if I pause for half a second?

Is it because you've never used copper pair telephone networks and only have used digital or cellular networks?

POTS is magical if you get end to end. Which I don't think is really a thing anymore. The last time I made a copper to copper call on POTS was in 2015! At&t was charging nearly $40 for that analog line per month so I shut it off. My VoIP line with long distance and international calling (the pots didn't) is $20/month with two phone numbers. And its routed through a PBX I control.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#192

Post by genewitch »

accrual wrote: post #129
flippy_flops wrote: post #120

Earlier quoted context omitted.

Need some vocal version of “heredoc”

"Hello AI, over", "Hello human, over". :) Oh, wait: "How do I iterate over a list-", "Iteration is a process where..." :p

We can recreate Shakma while we're at it with all the times we say "... Over."

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#193

Post by amelius »

diggan wrote: post #174
atoav wrote: post #169

Earlier quoted context omitted.

uv is the way. https://docs.astral.sh/uv/ Sadly it appears that people in the LLM space aren't really all that good at packaging their software (maybe, on purpose).

> Sadly it appears that people in the LLM space This seems to be somewhat of a Python side-effect, same goes for almost any Python projects thrown together by people who hasn't spent 10% of their life fighting dependency management in Python. But agree with uv being the best way. I'm not a "real" Python programmer, similar boat to parent that I just end up running a bunch of Python projects for various ML things, and…

For security maybe you should do all of this inside a sandbox.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#194

Post by genewitch »

sebra wrote: post #148
krick wrote: post #122

Cool for a weekend project, but honestly ChatGPT is still kinda shit at dialogues. I wonder if that's the issue with technology or OpenAI's fine-tuning (and suspect the latter), but it cannot talk like normal people do: shut up if it has nothing to add of value, ask reasonable follow-up questions if user doesn't understand something or there's ambiguity in the question. Also, on topic of follow-up questions: I don't…

I added this to personal instructions to make it less annoying: • No compliments, flattery, or emotional rapport. • Focus on clear reasoning and evidence. • Be critical of users assumptions when needed. • Ask follow-up questions only when essential for accuracy. However, I'm kinda concerned with crippling it by adding custom prompts. It's kinda hard to know how to use AI efficiently. But the glazing and random follow…

I often ask copilot about phrases I hear that I don't know or understand, like "what is a key party" - where I just want it to define it, and it will output three paragraphs that end with some suggestion that I am interested in it.

It is something that local models I have tried do not do, unless you are being conversational with it. I imagine openai gets a bit more pennies if they add the open ended questions to the end of every reply, and that's why it's done. I get annoyed if people patronize me, so too I get annoyed at a computer.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#195

Post by diggan »

amelius wrote: post #193
diggan wrote: post #174

Earlier quoted context omitted.

> Sadly it appears that people in the LLM space This seems to be somewhat of a Python side-effect, same goes for almost any Python projects thrown together by people who hasn't spent 10% of their life fighting dependency management in Python. But agree with uv being the best way. I'm not a "real" Python programmer, similar boat to parent that I just end up running a bunch of Python projects for various ML things, and…

For security maybe you should do all of this inside a sandbox.

I'm assuming most people run untrusted stuff like 3rd party libraries in some sort of isolated environment, unless they're begging to be hacked. Some basic security understanding has to be assumed, otherwise we have a long list to go through :)

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#196

Post by genewitch »

snet0 wrote: post #75
koljab wrote: post #29

Earlier quoted context omitted.

That's a great question! My first implementation was interruption on voice activity after echo cancellation. It still had way too many false positives. I changed it to incoming realtime transcription as a trigger. That adds a bit of latency but that gets compensated by way better accuracy. Edit: just realized the irony but it's really a good question lol

> That's a great question! Never forget what AI stole from us. This used to be a compliment, a genuine appreciation of a good question well-asked. Now it's tainted with the slimy, servile, sycophantic stink of AI chat models.

For at least 12 years it's been used as filler. Pay attention to interviews of any sort. Half the time it's in response to an obviously scripted question.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#197

Post by zamadatix »

zo1 wrote: post #186
sabellito wrote: post #164

Every time I see these things, they look cool as hell, I get excited, then I try to get them working on my gaming PC (that has the GPU), I spend 1-2h fighting with python and give up. Today's issue is that my python version is 3.12 instead of =3.9. Installing python 3.11 from the official website does nothing, I give up. It's a shame that the amazing work done by people like the OP gets underused because of this mess…

Do you know how much time I (or any other dev) would spend getting a C#, or C++, or JS/TS, or Java or any language project running that has anything to do with ML up and running on tech and tooling we are kinda unfamiliar with? Yes, pretty much 1-2 hours, and very likely more. Sorry but this sort of criticism is so contrived and low-effort. "Oh I tried compiling a language I don't know, using tooling I never use, usi…

For development is one thing, the problem is with this being how the LLM app distribution for general use works too.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#198

Post by whacked_new »

diggan wrote: post #174
atoav wrote: post #169

Earlier quoted context omitted.

uv is the way. https://docs.astral.sh/uv/ Sadly it appears that people in the LLM space aren't really all that good at packaging their software (maybe, on purpose).

> Sadly it appears that people in the LLM space This seems to be somewhat of a Python side-effect, same goes for almost any Python projects thrown together by people who hasn't spent 10% of their life fighting dependency management in Python. But agree with uv being the best way. I'm not a "real" Python programmer, similar boat to parent that I just end up running a bunch of Python projects for various ML things, and…

does uv play well with cuda?

I use nix-shell when possible to specify my entire dev environment (including gnumake, gcc, down to utils like jq)

it often doesn't play well with venv and cuda, which I get. I've succeeded in locking a cuda env with a nix flake exactly once, then it broke, and I gave up and went back to venv.

over the years I've used pip, pyenv, pip env, poetry, conda, mamba, younameit. there are always weird edge cases especially with publication codes that publish some intersection of a requirements.txt, pyproject.toml, a conda env, to nothing at all. There are always bizarro edge cases that make you forget if you're using python or node /snark

I'll be happy to use the final tool to rule them all but that's how they were all branded (even nix; and i know poetry2nix is not the way)

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#199

Post by bb88 »

whacked_new wrote: post #198
diggan wrote: post #174

Earlier quoted context omitted.

> Sadly it appears that people in the LLM space This seems to be somewhat of a Python side-effect, same goes for almost any Python projects thrown together by people who hasn't spent 10% of their life fighting dependency management in Python. But agree with uv being the best way. I'm not a "real" Python programmer, similar boat to parent that I just end up running a bunch of Python projects for various ML things, and…

does uv play well with cuda? I use nix-shell when possible to specify my entire dev environment (including gnumake, gcc, down to utils like jq) it often doesn't play well with venv and cuda, which I get. I've succeeded in locking a cuda env with a nix flake exactly once, then it broke, and I gave up and went back to venv. over the years I've used pip, pyenv, pip env, poetry, conda, mamba, younameit. there are always…

UV uses venv underneath.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#200

Post by mrinterweb »

sabellito wrote: post #164

Every time I see these things, they look cool as hell, I get excited, then I try to get them working on my gaming PC (that has the GPU), I spend 1-2h fighting with python and give up. Today's issue is that my python version is 3.12 instead of =3.9. Installing python 3.11 from the official website does nothing, I give up. It's a shame that the amazing work done by people like the OP gets underused because of this mess…

Meta comment about this thread: there is a lot of just use "x", use "y", use "z", use ... comments. Kind of proves the point of the top level comment.

I feel the same way when installing some python library. There's a bunch of ways to manage dependencies that I wish was more standardized.

Post reply on HN