We use this exact stack at work (OpenAI, ElevenLabs, Deepgram) for some exploratory use cases. The key issue we have now is latency with the LLM. Deepgram and Elevanlabs work brilliantly!
Jarvis: A Voice Virtual Assistant in Python (OpenAI, ElevenLabs, Deepgram)
21–30 of 60 posts
Re: Jarvis: A Voice Virtual Assistant in Python (OpenAI, ElevenLabs, Deepgram)
#22If you're really sitting at the computer for 24 hours...I have a family member that died from blood clots that formed when he sitting at his computer for too long.
Re: Jarvis: A Voice Virtual Assistant in Python (OpenAI, ElevenLabs, Deepgram)
#23We use this exact stack at work (OpenAI, ElevenLabs, Deepgram) for some exploratory use cases. The key issue we have now is latency with the LLM. Deepgram and Elevanlabs work brilliantly!
Great! What do you guys have in mind in terms of products using these tools. Yeah unfortunately it's hard to shave on latency.
Re: Jarvis: A Voice Virtual Assistant in Python (OpenAI, ElevenLabs, Deepgram)
#24Somewhat amusing to consider that the (in-character) Marvel Cinematic Universe JARVIS could have been an LLM!
And of course Ultron is an asshole, it was trained on input from Tony Stark!
Back in 2008/9 I wondered just what would be required to run JARVIS, something you could converse with naturally, would understand what you meant, and be able to take care of complex mechanical tasks. The Iron Man suits have always been mostly Do-What-I-Mean (DWIM) managed by JARVIS or other AI agents, and now all of that seems to be attainable.
It's going to be an interesting time discovering just how well a human and AI agent can work together. I could see a military personal spotter, keeping track of enemy combatants, managing larger awareness of the battlefield, etc. I wonder how much a soldier could safely offload?
Re: Jarvis: A Voice Virtual Assistant in Python (OpenAI, ElevenLabs, Deepgram)
#25Found Similar project but fully local for anyone interested. https://github.com/KoljaB/LocalAIVoiceChat This one looks similar to this but also does internet search, mail, music, smart home etc.. Hope that there is a standard interface that gets plugged into it. so anyone can develop addons. https://github.com/KoljaB/Linguflex
How did you find these? I was literally looking for tutorials all day long and could not find something. These projects look insane!
Yes, was searching for Realtime STT and got a hit on GitHub , then looked at his other projects and found he builts up on his STT and TTS projects, it’s just 2 second latency on Local voice chat. Which is very good .
Re: Jarvis: A Voice Virtual Assistant in Python (OpenAI, ElevenLabs, Deepgram)
#26Here I was thinking about putting something like this in my home, and jokingly calling it Jarvis. This will be a great starting point, shame you can't choose the models you want to talk to (ie use local models instead of OpenAI), but great nonetheless !
That was exactly my thought haha, I want Jarvis at home. You could easily modify my code to run a local LLM instead
Re: Jarvis: A Voice Virtual Assistant in Python (OpenAI, ElevenLabs, Deepgram)
#27Somewhat amusing to consider that the (in-character) Marvel Cinematic Universe JARVIS could have been an LLM! And of course Ultron is an asshole, it was trained on input from Tony Stark! Back in 2008/9 I wondered just what would be required to run JARVIS, something you could converse with naturally, would understand what you meant, and be able to take care of complex mechanical tasks. The Iron Man suits have always b…
Exactly my thought, I was like "Jarvis has got to be just a 2030 version of an LLM".
Yeah I actually considered making a spotter AI using computer vision in a game like ARMA 3 or Squad but kind of difficult. I made a spotter for ground vehicles on aerial imagery using YOLOv5 here: https://github.com/AlexandreSajus/Military-Vehicles-Image-Re...
There's a French defense company, Preligens, that actually does this currently
Re: Jarvis: A Voice Virtual Assistant in Python (OpenAI, ElevenLabs, Deepgram)
#28Here I was thinking about putting something like this in my home, and jokingly calling it Jarvis. This will be a great starting point, shame you can't choose the models you want to talk to (ie use local models instead of OpenAI), but great nonetheless !
That was exactly my thought haha, I want Jarvis at home. You could easily modify my code to run a local LLM instead
Actually there are a few LLM wrappers around that use the openai API spec (localai is a good one)... so you could just allow a configurable openai endpoint URI and technically users can swap in any model.
Re: Jarvis: A Voice Virtual Assistant in Python (OpenAI, ElevenLabs, Deepgram)
#29Here is a video demo of the project: https://youtu.be/aIg4-eL9ATc?si=66ynl4Mlci9v76rU
Nice work! Very impressed. Do you happen to know anything about any open source voice identification software? I’ve noticed with ChatGPT voice and any other voice driven assistant that a massive problem is the background voices and noise. One solution could be advanced pre-processing to ID your voice only. Another idea I’ve had is using something professional with PTT: https://sheepdogmics.com/products/quick-disconne…
Check whether this can help https://github.com/resemble-ai/resemble-enhance/tree/main
Re: Jarvis: A Voice Virtual Assistant in Python (OpenAI, ElevenLabs, Deepgram)
#30I wonder if we're at a point where you could build a voice assistant like that, except almost-realtime and streamed end to end: User speaks and speech to text starts streaming text while the user is still speaking. That text stream is piped into a LLM, which also streams its output text. That output text is streamed to text-to-speech, which also generates audio in a streaming manner.
Any existing stream api for llm input?