Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Real-time AI Voice Chat at ~500ms Latency

github.com

Post reply on HN
Print view View original

201–210 of 238 posts

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#201

Post by bb88 »

diggan wrote: post #174
atoav wrote: post #169

Earlier quoted context omitted.

uv is the way. https://docs.astral.sh/uv/ Sadly it appears that people in the LLM space aren't really all that good at packaging their software (maybe, on purpose).

> Sadly it appears that people in the LLM space This seems to be somewhat of a Python side-effect, same goes for almost any Python projects thrown together by people who hasn't spent 10% of their life fighting dependency management in Python. But agree with uv being the best way. I'm not a "real" Python programmer, similar boat to parent that I just end up running a bunch of Python projects for various ML things, and…

There's been a movement away from requirements.txt towards pyproject.toml. And commands like "uv add" and "uv install" take most of the pain of initializing and maintaining those dependencies.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#202

Post by regularfry »

underdeserver wrote: post #177
regularfry wrote: post #176

Earlier quoted context omitted.

Virtual environments with venv don't answer the python version problem unless you throw another tool into the mix.

Hmm? My venvs do include the Python version (via symlink to /bin). Don't yours?

The GP's problem was (apparently) an inability to install the right python version, not an inability to select it.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#203

Post by amelius »

diggan wrote: post #195
amelius wrote: post #193

Earlier quoted context omitted.

For security maybe you should do all of this inside a sandbox.

I'm assuming most people run untrusted stuff like 3rd party libraries in some sort of isolated environment, unless they're begging to be hacked. Some basic security understanding has to be assumed, otherwise we have a long list to go through :)

Ok, but getting your GPU to work inside a sandbox can be a difficult step too. I bet most people give up and just run the commands without a sandbox.

Therefore, maybe it is a good idea to include those instructions.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#204

Post by WanderPanda »

s5ma6n wrote: post #184
jhoho wrote: post #167

Earlier quoted context omitted.

Let me introduce you to the beautiful world of virtual environments. They save you the headache of getting a full installation to run, especially when using Windows. I prefer miniconda, but venv also does the job.

I am also using conda and specifically mamba which has a really quick dependency solver. However, sometimes repos require system level packages as well. Tried to run TRELLIS recently and gave up after 2h of tinkering around to get it to work in Windows. Also, whenever I try to run some new repo locally, creating a new virtual environment takes a ton of disk space due to CUDA and PyTorch libraries. It adds up quickly…

Same experience. They should really store these blobs centrally under a hash and link to them from the venvs

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#205

Post by reginald78 »

sabellito wrote: post #164

Every time I see these things, they look cool as hell, I get excited, then I try to get them working on my gaming PC (that has the GPU), I spend 1-2h fighting with python and give up. Today's issue is that my python version is 3.12 instead of =3.9. Installing python 3.11 from the official website does nothing, I give up. It's a shame that the amazing work done by people like the OP gets underused because of this mess…

I spin up a whole linux VM with a passed through nvidia GPU for these and I still spend the majority of the time fighting the python and figuring out the missing steps in the setup instructions.

Glad for this thread though since it looks like there's some tricks I haven't tried, plus since it seems a lot of other people have similar issues I feel less dumb.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#206

Post by pokechamp »

sabellito wrote: post #164

Every time I see these things, they look cool as hell, I get excited, then I try to get them working on my gaming PC (that has the GPU), I spend 1-2h fighting with python and give up. Today's issue is that my python version is 3.12 instead of =3.9. Installing python 3.11 from the official website does nothing, I give up. It's a shame that the amazing work done by people like the OP gets underused because of this mess…

[flagged]

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#207

Post by thekaranchawla »

koljab wrote: post #85
kabirgoel wrote: post #69

This is great. Poking into the source, I find it interesting that the author implemented a custom turn detection strategy, instead of using Silero VAD (which is standard in the voice agents space). I’m very curious why they did it this way and what benefits they observed. For folks that are curious about the state of the voice agents space, Daily (the WebRTC company) has a great guide [1], as well as an open-source f…

It's in fact using Silero via RealtimeSTT. RealtimeSTT tells when silence starts. Then a binary sentence classification model is used on the realtime transcription text which infers blazingly fast (10ms) and returns a probability between 0 and 1 indicating if the current spoken sentence is considered "complete". The turn detection component takes this information to calculate the silence waiting time until "turn is o…

This is the exact strategy I'm using for the real-time voice agent I'm building. Livekit also published a custom turn detection model that works really well based on the video they released, which was cool to see.

Code: https://github.com/livekit/agents/tree/main/livekit-plugins/... Blog: https://blog.livekit.io/using-a-transformer-to-improve-end-o...

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#208

Post by mckeed »

jhoho wrote: post #167
sabellito wrote: post #164

Every time I see these things, they look cool as hell, I get excited, then I try to get them working on my gaming PC (that has the GPU), I spend 1-2h fighting with python and give up. Today's issue is that my python version is 3.12 instead of =3.9. Installing python 3.11 from the official website does nothing, I give up. It's a shame that the amazing work done by people like the OP gets underused because of this mess…

Let me introduce you to the beautiful world of virtual environments. They save you the headache of getting a full installation to run, especially when using Windows. I prefer miniconda, but venv also does the job.

As someone who doesn't develop in python but occasionally tries to run python projects, it's pretty annoying to have to look up how to use venv every time.

I finally added two scripts to my path for `python` and `pip` that automatically create and activate a virtual env at `./.venv` if there isn't one active already. It would be nice if something like that was just built into pip so there could be a single command to run like Ruby has now with Bundler.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#209

Post by homeless_engi »

regularfry wrote: post #176
jhoho wrote: post #167

Earlier quoted context omitted.

Let me introduce you to the beautiful world of virtual environments. They save you the headache of getting a full installation to run, especially when using Windows. I prefer miniconda, but venv also does the job.

Virtual environments with venv don't answer the python version problem unless you throw another tool into the mix.

Conda does! `conda create -n myenv python=3.9`, for example

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#210

Post by diggan »

whacked_new wrote: post #198
diggan wrote: post #174

Earlier quoted context omitted.

> Sadly it appears that people in the LLM space This seems to be somewhat of a Python side-effect, same goes for almost any Python projects thrown together by people who hasn't spent 10% of their life fighting dependency management in Python. But agree with uv being the best way. I'm not a "real" Python programmer, similar boat to parent that I just end up running a bunch of Python projects for various ML things, and…

does uv play well with cuda? I use nix-shell when possible to specify my entire dev environment (including gnumake, gcc, down to utils like jq) it often doesn't play well with venv and cuda, which I get. I've succeeded in locking a cuda env with a nix flake exactly once, then it broke, and I gave up and went back to venv. over the years I've used pip, pyenv, pip env, poetry, conda, mamba, younameit. there are always…

I generally use nix-shell whenever I can too, only resorting to `uv` for projects where I cannot expect others to neccessarly understand Nix enough to handle the nix-shell stuff, even if it's trivial for me.

AFAIK, it works as well with cuda as any other similar tool. I personally haven't had any issues, most recently last week I was working on a transformer model for categorizing video files and it's all managed with uv and pytorch installed into the venv as normal.

Post reply on HN