31–40 of 60 posts
We use this exact stack at work (OpenAI, ElevenLabs, Deepgram) for some exploratory use cases. The key issue we have now is latency with the LLM. Deepgram and Elevanlabs work brilliantly!
"Jarvis" is a trademark of Marvel, so that name will definitely not work. https://trademarks.justia.com/862/94/jarvis-86294162.html
But there are many apps called Jarvis, so I am not sure how that is supposed to work?
"Jarvis" is a trademark of Marvel, so that name will definitely not work. https://trademarks.justia.com/862/94/jarvis-86294162.html
Uh oh I hope I'm not in trouble
Somewhat amusing to consider that the (in-character) Marvel Cinematic Universe JARVIS could have been an LLM! And of course Ultron is an asshole, it was trained on input from Tony Stark! Back in 2008/9 I wondered just what would be required to run JARVIS, something you could converse with naturally, would understand what you meant, and be able to take care of complex mechanical tasks. The Iron Man suits have always b…
Exactly my thought, I was like "Jarvis has got to be just a 2030 version of an LLM". Yeah I actually considered making a spotter AI using computer vision in a game like ARMA 3 or Squad but kind of difficult. I made a spotter for ground vehicles on aerial imagery using YOLOv5 here: https://github.com/AlexandreSajus/Military-Vehicles-Image-Re... There's a French defense company, Preligens, that actually does this curre…
More of a framework to perform the general purpose task of "recognize things in 30 (60? 120?) frames per second video and act on events in the video"
I'm curious - what made you choose deepgram over just running whisper? I don't have any experience with deepgram but whisper has worked so well in my own tests that I didn't even ever consider there might be API speech recognition-only companies.
I'm curious - what made you choose deepgram over just running whisper? I don't have any experience with deepgram but whisper has worked so well in my own tests that I didn't even ever consider there might be API speech recognition-only companies.
Deepgram advertised itself as being the fastest, and I wanted to focus on limiting response delay so I chose it. I hope I did not get misled.
We use this exact stack at work (OpenAI, ElevenLabs, Deepgram) for some exploratory use cases. The key issue we have now is latency with the LLM. Deepgram and Elevanlabs work brilliantly!
Great! What do you guys have in mind in terms of products using these tools. Yeah unfortunately it's hard to shave on latency.
That live broadcast created a lot of buzz and numerous other use cases have popped up across the company. I'm working on a tech blog showcase next week to show it off on HN hopefully!
I wonder if we're at a point where you could build a voice assistant like that, except almost-realtime and streamed end to end: User speaks and speech to text starts streaming text while the user is still speaking. That text stream is piped into a LLM, which also streams its output text. That output text is streamed to text-to-speech, which also generates audio in a streaming manner.
It wasn't really streamed, though. Audio input was buffered, fully evaluated to a string, then fed into the LLM and the full text was converted back to audio.
The Whisper speech-to-text was pretty real-time, the LLM was not. I was barely scraping by on hardware specs, though.
I wonder if we're at a point where you could build a voice assistant like that, except almost-realtime and streamed end to end: User speaks and speech to text starts streaming text while the user is still speaking. That text stream is piped into a LLM, which also streams its output text. That output text is streamed to text-to-speech, which also generates audio in a streaming manner.
You can do the "almost-realtime" part, all locally. I tinkered with a Python script for a few hours that used Whisper to speech-to-text, fed that into a local Mistral model (don't recall which), and then piped the output into text-to-speech. It wasn't really streamed, though. Audio input was buffered, fully evaluated to a string, then fed into the LLM and the full text was converted back to audio. The Whisper speech-…
I wonder if we're at a point where you could build a voice assistant like that, except almost-realtime and streamed end to end: User speaks and speech to text starts streaming text while the user is still speaking. That text stream is piped into a LLM, which also streams its output text. That output text is streamed to text-to-speech, which also generates audio in a streaming manner.