Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

news.ycombinator.com

Post reply on HN
Print view View original

21–30 of 71 posts

Re: Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

#21

Post by euvin »

sumanyusharma wrote: post #19
euvin wrote: post #17

The idea of testing an agent with annoying situations, like uncooperative people or vague responses, makes me wonder if, in the future, similar approaches might be tried on humans. People could be (unknowingly) subjected to automated "social benchmarks" with artificially designed situations, which I'm sure I don't have to explain how dystopian that is. It would essentially be another form of a behavioral interview. I…

I wonder if a more optimistic version of this could be used to train humans and improve their skills. I'm thinking along the lines of LeetCode / Project Euler, but more dynamic and personalized! Few examples: 1) Customer service: Simulating challenging customer interactions could help reps develop patience and problem-solving skills. 2) Emergency responders: Creating realistic crisis scenarios (like 911 calls) that c…

That does sound like an interesting idea. Upon further thought, I think that it would heavily depend on implementation.

In a bad case, I envision a ton of companies or institutions employing very strict & narrow situations to the point where they only accept a very homogenized personality. It could end up creating a stiff or worse culture than if they had naturally accumulated a diverse population, if that makes sense. Discrimination already exists, but would be made a lot easier, automated, and commonplace.

In a good case, extremely antisocial behavior (situations that are "softballs" or "hard to screw up for reasonable people") could be easily caught at scale and addressed an early age. Plus the cases you've listed, eliminating the need for special attention and mentorship from the limited people we meet irl.

I'm sure there are other horrible or amazing cases I'm missing.

So as all tools are, it would depend. Whether this will actually benefit more than harm will depend on the society you place it in, and I'm not sure I have that much faith in the corporate world.

Re: Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

#22

Post by sumanyusharma »

atyro wrote: post #18

Nice! Great to see the UI looks clean enough that it's accessible to non-engineers. The prompt management and active monitoring combo looks especially useful. Been looking for something with this combo for an expense app we're building.

Yes! We're aiming to build a tool that both engineers and non-engineers love.

We've discovered that it's often faster for non-technical domain experts to iterate on prompts in a structured, eval-driven way, rather than relying on engineers to translate business requirements into prompts.

While storing prompts in code offers version control benefits, it can hinder collaboration. On the other hand, using a pure CMS for prompts enhances collaboration but sacrifices some modern software development practices.

We're working towards a solution that bridges this gap, combining the best of both approaches. We're not there yet, but we have a clear roadmap to achieve this vision!

Re: Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

#23

Post by serjester »

I feel like the better positioning would be evals for voice agents. It seems just as challenging to figure out all the ways your system can go wrong, as it is to build the system in the first place. Doing this in a way that actually adds value without any domain expertise, seems impossible.

If it did, wouldn't all the companies with production AI text interfaces be using similar techniques? Now being able to easily replay a conversation that was recorded with a real user seems like a huge value add.

Re: Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

#25

Post by sumanyusharma »

serjester wrote: post #23

I feel like the better positioning would be evals for voice agents. It seems just as challenging to figure out all the ways your system can go wrong, as it is to build the system in the first place. Doing this in a way that actually adds value without any domain expertise, seems impossible. If it did, wouldn't all the companies with production AI text interfaces be using similar techniques? Now being able to easily r…

Absolutely agree that creating effective evals requires domain expertise. Right now, we're co-building evals with customers, but we're identifying which aspects can be productized.

Regarding text-based evals — part of testing voice agents involves assessing their core reasoning logic. To do that, we bypass the voice layer and simulate conversations via text. So yes, the core simulation engine is reusable for both conversational text and voice interactions.

We're also excited about shipping the ability to replay a simulated conversation inspired by a real user!

Re: Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

#26

Post by bongodongobob »

meiraleal wrote: post #16

There is not even one reliable and proven "voice agent" yet (correct me if I'm wrong but the best available, elevenlabs, isn't that great yet to be a voice agent) but there is already companies selling the test of voice agents? Selling shovels on a gold rush seems to have become the only one mantra here.

As a test, I asked GPT to call my phone company and get my account balance. It worked and even declined some program they tried to sign me up for. Blew my mind.

Re: Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

#27

Post by esafak »

zebomon wrote: post #10

As someone whose job has been negatively impacted by LLMs already, I'll echo the sentiment here that use cases like this one are sort of depressing, as they will primarily impact people who work long hours for small pay. It certainly seems like there's money to be made in this, so congratulations. The landing page is clear and inviting as well. I think I understand what my workflow inside it would be like based on yo…

What line of work was it?

Re: Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

#28

Post by Matticus_Rex »

euvin wrote: post #7
Matticus_Rex wrote: post #5

Earlier quoted context omitted.

Well, the idea behind this product is to make sure the LLMs they replace/augment workers with aren't total dogshit (at least relative to the workers they're replacing). But also remember -- the point of the economy is not jobs. It's value creation. If we can create the same or greater value with fewer people working/people working less, that's a great result! And it's the result we've seen continue over the last cent…

I'm speaking as a potentially ignorant layman here (in the US), but if you need a job for income then isn't that taking away a lot of avenues for entry career progression and way more competition for remaining roles? I don't think reducing the need for human labor is inherently bad, but our current society seems to be heavily centered around finding work.

At least at the moment, the jobs we're talking about (e.g. drive-through order-taker) aren't meaningful entry into a career path — they're jobs people take before they have a career path, or on a largely short-term basis. There are exceptions, but there are still going to be plenty of restaurant jobs for a while. We're still in a shortage of labor on that end of the job market.

But also, the number of remaining roles isn't fixed. Jobs exist (at least in the private sector) because they create more value than they cost to fill, and we're always finding new and expanded ways for people to create more value. Saving resources through automation just means we can redirect that value creation somewhere else.

Ultimately this is how economies grow and the world becomes wealthier over time; we increase the value of people's time because there's so much competition for it, to the point where we can then more cheaply automate some or part of the job. If the supply of labor gets too large for the uses we can find for it, prices for labor fall, and the relative cost of automation is increased.

Re: Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

#29

Post by michaelmior »

themacguffinman wrote: post #20

AI voice agents are weird to me because voice is already a very inefficient and ambiguous medium, the only reason I would make a voice call is to talk to a human who is equipped to tackle the ambiguous edge cases that the engineers didn't already anticipate. If you're going to develop AI voice agents to tackle pre-determined cases, why wouldn't you just develop a self-serve non-voice UI that's way more efficient? Why…

I'm the same way, and I don't have any data on this, but it's possible that we're in the minority. This probably isn't the case, but hopefully anyone implementing such a system has thought through whether it will actually provide any value.

For example, if you had an existing IVR system and you tracked menu options and found that a significant portion of calls were able to be answered by non-smart pre-recorded messages, upgrading to an AI voice agent could be a reasonable improvement.

Re: Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents

#30

Post by zebomon »

esafak wrote: post #27
zebomon wrote: post #10

As someone whose job has been negatively impacted by LLMs already, I'll echo the sentiment here that use cases like this one are sort of depressing, as they will primarily impact people who work long hours for small pay. It certainly seems like there's money to be made in this, so congratulations. The landing page is clear and inviting as well. I think I understand what my workflow inside it would be like based on yo…

What line of work was it?

I'm a Top Rated/Pro-verified ghostwriter on Fiverr. It's been my full-time job since 2015. Went from mid-six figures in 2022 to scraping by today.
Post reply on HN