Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

news.ycombinator.com

Post reply on HN
Print view View original

61–70 of 71 posts

Re: Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

#61

Post by yuppiepuppie »

jcims wrote: post #44
themacguffinman wrote: post #20

AI voice agents are weird to me because voice is already a very inefficient and ambiguous medium, the only reason I would make a voice call is to talk to a human who is equipped to tackle the ambiguous edge cases that the engineers didn't already anticipate. If you're going to develop AI voice agents to tackle pre-determined cases, why wouldn't you just develop a self-serve non-voice UI that's way more efficient? Why…

Think 1-800-CONTACTS not Siri. Call centers are super expensive and the user experience is usually pretty bad. There's a huge incentive to move to voice agents, but one of the challenges is building a framework to adequately test it. That seems to be what this is focused on.

If I understand correctly, it’s the push back on the call center in general when using AI agents. Why go through the trouble at that point versus another manner of fixing my issue.

For example, when I need to activate a new SIM card, I need to call the company to get it activated. But if I’m talking to an AI agent at that point, why not have me go through another channel (website/app?) to activate it?

Re: Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

#62

Post by pj_mukh »

sumanyusharma wrote: post #53
pj_mukh wrote: post #12

My 2.5 year old yesterday starting saying "Hey, This is a test, Can you hear me?", parroting me spending hours testing my LLM. Hah. This will work with a https://www.pipecat.ai type system? Would love to wrap a continuous testing system with my bot.

Pipecat looks awesome! I'll run the examples over the weekend and try to see what the integration hooks need to look like: https://github.com/pipecat-ai/pipecat/tree/main/examples It should be pretty straightforward at first glance!

Yea, it's interesting, it's just a Chatbot over a Zoom (we use Daily) call as opposed to a 1-on-1 websocket (or a phone call). Other advantage is using WebRTC!

Re: Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

#63

Post by sumanyusharma »

bazlan wrote: post #55

As someone who has worked in TTS for over 4 years now. I can tell you that evaluation is the most difficult aspect of generative audio ML. How will this really check that the models are performing well vs just listening?

We're focused on end-to-end evals focused on function-call accuracy, style, tone & latency of the conversations between our sims and your voice agent. Less focused on pure TTS evals at the moment!

Re: Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

#65

Post by Centigonal »

smeej wrote: post #59
sumanyusharma wrote: post #41

Earlier quoted context omitted.

I use Superwhisper (no affiliation, just a happy user), which runs a local Whisper model, to create most of my email drafts and post-meeting notes. I find Whisper more accurate than Mac’s built-in speech-to-text, plus I’m faster at speaking than typing. Sometimes, I even ‘talk’ into Cursor’s chat window instead of typing. The only downside? It can get a bit annoying for others when you're talking to yourself all day.

I'm looking for something like this that runs on Linux. Best thing I've found is LiveCaptions, but its output is janky. I can't just use it to type in any old text field, and its output requires substantial editing after the fact. I guess I understand that a lot of things are being developed for Apple silicon specifically. It's just frustrating that despite hours of searching, I'm not finding anything decent.

Talon Voice is good and runs on linux.

https://talonvoice.com/

Re: Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

#66

Post by fakedang »

sumanyusharma wrote: post #19
euvin wrote: post #17

The idea of testing an agent with annoying situations, like uncooperative people or vague responses, makes me wonder if, in the future, similar approaches might be tried on humans. People could be (unknowingly) subjected to automated "social benchmarks" with artificially designed situations, which I'm sure I don't have to explain how dystopian that is. It would essentially be another form of a behavioral interview. I…

I wonder if a more optimistic version of this could be used to train humans and improve their skills. I'm thinking along the lines of LeetCode / Project Euler, but more dynamic and personalized! Few examples: 1) Customer service: Simulating challenging customer interactions could help reps develop patience and problem-solving skills. 2) Emergency responders: Creating realistic crisis scenarios (like 911 calls) that c…

There's already a startup for the last use case. I forgot the name though.

Re: Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

#67

Post by smeej »

Centigonal wrote: post #65
smeej wrote: post #59

Earlier quoted context omitted.

I'm looking for something like this that runs on Linux. Best thing I've found is LiveCaptions, but its output is janky. I can't just use it to type in any old text field, and its output requires substantial editing after the fact. I guess I understand that a lot of things are being developed for Apple silicon specifically. It's just frustrating that despite hours of searching, I'm not finding anything decent.

Talon Voice is good and runs on linux. https://talonvoice.com/

This looks really powerful for controlling the system with different scripts, but what if all I want it to do is let me narrate something and print out the sentences as close to real-time as possible? It's really just good STT that I'm looking for out of it.

Re: Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

#68

Post by vladsanchez »

smeej wrote: post #58
themacguffinman wrote: post #20

AI voice agents are weird to me because voice is already a very inefficient and ambiguous medium, the only reason I would make a voice call is to talk to a human who is equipped to tackle the ambiguous edge cases that the engineers didn't already anticipate. If you're going to develop AI voice agents to tackle pre-determined cases, why wouldn't you just develop a self-serve non-voice UI that's way more efficient? Why…

The example of "fast food drive-thru" really cleared this up for me. Frankly I'm surprised there isn't already some sort of NFC info transfer system in fast food restaurants' apps that lets you and everyone in your car enter your order while you're waiting in line, then knows when your car is up and brings you the food. Have the voice part be a fallback tier, not the primary one. My grocery store can know when I'm ar…

It's been a dream of mine for some years now. They're pretty dumb still today.

Re: Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

#69

Post by Centigonal »

smeej wrote: post #67
Centigonal wrote: post #65

Earlier quoted context omitted.

Talon Voice is good and runs on linux. https://talonvoice.com/

This looks really powerful for controlling the system with different scripts, but what if all I want it to do is let me narrate something and print out the sentences as close to real-time as possible? It's really just good STT that I'm looking for out of it.

The Talon voice dev created his own STT model that's very performant. The transcription quality is... good, but not world-class. It's better than anything that came out before Whisper IMO, but the newest generator of models can do things like inferring punctuation and words outside of its vocabulary (although the downside of the new generation of VTT is that they can sometimes hallucinate words that are very different from what you said).

It's a bit overkill to use Talon for just voice dictation, but that is 90% of what I use it for, and it's pretty good at it.

Re: Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

#70

Post by telecomhacker »

sumanyusharma wrote: post #40
telecomhacker wrote: post #39

I work in the telecom space. I don't think this paradigm will get adopted in the near future. Customers are already building voice bots on top of Google Dialogflow e.g. Cognigy. Cognigy does have LLM capabilities, but it is not widely adopted. I think voice bots will still have to be manually configured for some time.

I'm curious to learn more about what's blocking the widespread adoption of the LLM capabilities. Lack of knowledge, reliability, or something else?

You guys should try and get acquired by Cognigy.
Post reply on HN