Launch HN: Roark (YC W25) – Taking the pain out of voice AI testing

Hey HN, we’re James and Daniel, co-founders of Roark (https://roark.ai). We built a tool that lets developers replay real production calls against their latest Voice AI changes, so they can catch failures, test updates, and iterate with confidence.

Here’s a demo video: https://www.youtube.com/watch?v=eu8mo28LsTc.

We ran into this problem while building a voice AI agent for a dental clinic. Patients kept getting stuck in loops, failing to confirm insurance, or misunderstanding responses. The only way to test fixes was to manually call the agent or read through hundreds of transcripts, hoping to catch issues. It was slow, frustrating, and unreliable.

Talking to other teams, we found this wasn’t just a niche issue - every team building Voice AI struggled to validate performance efficiently. Debugging meant calling the agent over and over. Updates shipped with unknown regressions. Sentiment analysis relied only on text, missing key audio cues like hesitation or frustration, which often signal deeper issues.

That’s why we built Roark. Instead of relying on scripted test cases, Roark captures real production calls from VAPI, Retell, or a custom-built agent via API and replays them against your latest agent changes. We don’t just feed back text, we preserve what the user said, how they said it, and when they said it, mimicking pauses, sentiment, and tone up until the conversation flow changes. This ensures your agent is tested under real-world conditions, not just synthetic scripts.

For each replay that we run, Roark checks if the agent follows key flows (e.g. verifying identity before sharing account details). Our speech based evaluators also detect sentiments such as frustration and confusion, long pauses, and interruptions - things that regular transcripts miss.

After testing, Roark provides Mixpanel-style analytics to track failures, conversation flows, and key performance metrics, helping teams debug faster and ship with confidence. Instead of hoping changes work, teams get immediate pass/fail results, side-by-side transcript comparisons, and real-world insights.

We’re already working with teams in healthcare, legal, and customer service who rely on Voice AI for critical interactions. They use Roark to debug AI failures faster, test updates before they go live, and improve customer experiences - without manually calling their bots dozens of times.

Our product isn’t quite ready yet for self-service, so you’ll still see the dreaded “book a demo” on our home page. If you’re reading this, though, we’d love to fast-track you, so we made a special page for HN signups here: https://roark.ai/hn-access. If you’re working on Voice AI and want to try us out, please do!

Would love any feedback, thoughts, or questions from the HN community!

Appreciate the feedback! Completely agree - authentication should be handled at the system level, not just in prompts. This demo is meant to showcase how teams can build test cases from real failures and ensure fixes work before deployment. We’ll consider using a better example.

Your post suggests authorization as a feature:

> For each replay that we run, Roark checks if the agent follows key flows (e.g. verifying identity before sharing account details)

I don't know if AI will be more susceptible or less susceptible to phishing than humans, but this feels like a bad practice.

Thanks, Jan! Would love to hear more about what you’re working on! Just sent you an email.

Looks somewhat useful for voice AI QA.

But I wonder if a company is deploying voice AI, wouldn't they have their own testing and quality assurance flows?

Is this targeted at companies without an engineering department or something? In which case I find it surprising they're able to slot in some voice AI assistant in the first place.