Live data from Hacker News

Show HN: Real-time AI Voice Chat at ~500ms Latency

github.com

171–180 of 238 posts

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#171
post #35

I did some research into this about a year ago. Some fun facts I learned: - The median delay between speakers in a human to human conversation is zero milliseconds. In other words, about 1/2 the time, one speaker interrupts the other, making the delay negative. - Humans don't care about delays when speaking to known AIs. They assume the AI will need time to think. Most users will qualify a 1000ms delay is acceptable…

> Humans don't care about delays when speaking to known AIs.

I do care. Although 500ms is probably fine. But anything longer feels extremely clunky to the point of not being worth using.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#174
post #169

Every time I see these things, they look cool as hell, I get excited, then I try to get them working on my gaming PC (that has the GPU), I spend 1-2h fighting with python and give up. Today's issue is that my python version is 3.12 instead of =3.9. Installing python 3.11 from the official website does nothing, I give up. It's a shame that the amazing work done by people like the OP gets underused because of this mess…

uv is the way. https://docs.astral.sh/uv/ Sadly it appears that people in the LLM space aren't really all that good at packaging their software (maybe, on purpose).

> Sadly it appears that people in the LLM space

This seems to be somewhat of a Python side-effect, same goes for almost any Python projects thrown together by people who hasn't spent 10% of their life fighting dependency management in Python.

But agree with uv being the best way. I'm not a "real" Python programmer, similar boat to parent that I just end up running a bunch of Python projects for various ML things, and also create some smaller projects myself. Tried conda, micromamba, uv, and a bunch of stuff in-between, most of them breaks at one point or another, meanwhile uv gives me the two most important things in one neat package: Flexible Python versions depending on project, and easy management of venv's.

So for people who haven't given it a try yet, do! It does make using Python a lot easier when it comes to dependencies. These are the commands I tend to use according to my history, maybe it's useful as a sort of quickstart. I started using uv maybe 6 months, and this is a summary of literally everything I've used it for so far.

    # create new venv in working directory with pip + specific python version
    uv venv --seed --python=3.10
    # activate the venv
    source .venv/bin/activate
    # on-the-fly install pip dependencies
    uv pip install transformers
    # write currently installed deps to file
    uv pip freeze > requirements.txt
    # Later...
    # install deps from file
    uv pip install -r requirements.txt
    # run arbitrary file with venv in path etc
    uv run my_app.py
    # install a "tool" (like global CLIs) with a specific python version, and optional dependency version
    uv tool install --force --python python3.12 aider-chat@latest

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#175

Saying this as a user of these tools (openai, Google voice chat etc). These are fast yes, but they don't allow talking naturally with pauses. When we talk, we take long and small pauses for thinking or for other reasons. With these tools, AI starts taking as soon as we stop. Happens both in text and voice chat tools. I saw a demo on twitter a few weeks back where AI was waiting for the person to actually finish what…

Pauses are good as a first indicator, but when a pause occurs then what's been said so far should be fed to the model to decide if it's time to chip in or wait a bit for more.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#176
post #167

Every time I see these things, they look cool as hell, I get excited, then I try to get them working on my gaming PC (that has the GPU), I spend 1-2h fighting with python and give up. Today's issue is that my python version is 3.12 instead of =3.9. Installing python 3.11 from the official website does nothing, I give up. It's a shame that the amazing work done by people like the OP gets underused because of this mess…

Let me introduce you to the beautiful world of virtual environments. They save you the headache of getting a full installation to run, especially when using Windows. I prefer miniconda, but venv also does the job.

Virtual environments with venv don't answer the python version problem unless you throw another tool into the mix.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#177
post #167

Earlier quoted context omitted.

Let me introduce you to the beautiful world of virtual environments. They save you the headache of getting a full installation to run, especially when using Windows. I prefer miniconda, but venv also does the job.

Virtual environments with venv don't answer the python version problem unless you throw another tool into the mix.

Hmm? My venvs do include the Python version (via symlink to /bin). Don't yours?

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#178

Earlier quoted context omitted.

Virtual environments with venv don't answer the python version problem unless you throw another tool into the mix.

Hmm? My venvs do include the Python version (via symlink to /bin). Don't yours?

If you use something like uv (expanded here: https://news.ycombinator.com/item?id=43904078), I think it does. But if you just do `python -m venv .venv`, you get the specific version you used to create the virtual environment with. Some OSes seem to distribute binaries like `python3.8`, `python3.9` and so on so you could do `python3.8 -m venv .venv` to look one env to a specific version, but a bit of a hassle.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#179
post #143

In the demo, is there any specific reason that the voice doesn't go "up" in pitch when asking questions? Even the (many) rethorical questions would in my view improve by having a bit of a pitch change before the question mark.

There’s no SSML. The model that came up with the text knows what it’s saying in theory and therefore would know that it’s a question, if the mood should be sombre or excited and then can pass this information as SSML tags to the text to speech synthesizer. The problem I’ve been seeing is that pretty much all of these models are just outputting text and the text is being shoved into the TTS. It’s on my list to look into projects that have embedded these tags so that on the one hand you have like open web UI that’s showing a user text, but there’s actually an embedded set of tags that are handled by the TTS so that it sounds more natural. This project looks hackable for that purpose.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#180

Every time I see these things, they look cool as hell, I get excited, then I try to get them working on my gaming PC (that has the GPU), I spend 1-2h fighting with python and give up. Today's issue is that my python version is 3.12 instead of =3.9. Installing python 3.11 from the official website does nothing, I give up. It's a shame that the amazing work done by people like the OP gets underused because of this mess…

I've had a lot of success using podman with pyenv for python versions, and plain old venv for the actual environment. All of this lives within WSL, but you can still access everything locally with localhost://

If you just want to use windows, pyenv-win exists and works pretty well; just set a local version, then instantiate your venv.

uv does certainly feel like the future, but I have no interest in participating in a future VC rugpull.

Post reply on HN