Live data from Hacker News

Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

agnost.ai

41–50 of 63 posts

Re: Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

#41
post #40

Earlier quoted context omitted.

yea, at our volume which we still consider small as we've been able to figure out a way with llms & embeddings, its still fine. + we onboarded a voice ai company with more than 2 hour calls and thats when it was super hard to solve since there were so many elements to consider. model drifting is something a lot of folks do face after 5th/6th turn as per my understanding and it usually the median, how did you tackle i…

To clarify, I wasn't criticizing your approach or product, more responding to the people dismissing the problem you are solving. Regarding my experience, I have done a fair amount of work in the contact center space with long calls. I used statistical Bayesian approaches which I found to be much more resilient especially on long documents than embeddings/transformers. It also provided a joint modeling foundation for…

im hearing this for the first time and damn! i just told this to my cofounder/cto and he said hes gonna give this a shot in the coming days.

damn, i read bayesian in statistics like years ago, never thought itll come back this way

Re: Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

#43
post #40

Earlier quoted context omitted.

To clarify, I wasn't criticizing your approach or product, more responding to the people dismissing the problem you are solving. Regarding my experience, I have done a fair amount of work in the contact center space with long calls. I used statistical Bayesian approaches which I found to be much more resilient especially on long documents than embeddings/transformers. It also provided a joint modeling foundation for…

im hearing this for the first time and damn! i just told this to my cofounder/cto and he said hes gonna give this a shot in the coming days. damn, i read bayesian in statistics like years ago, never thought itll come back this way

Happy to chat more in depth if more details would be helpful. I think my contact info is accessible from my HN profile.

Re: Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

#45
Why... why do companies keep taking every tiny feature and trying to productize it?

In the tradition of boring software, even before LLMs it was much simpler to just use your existing tools and hand-roll. With LLMs I cannot fathom reaching for a product for something small like this.

Re: Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

#46
post #30

I see a fair number of comments here advocating for either codex to hand-roll this themselves, or to simply punt to SQL. I do want to advocate for the difficulty of the problem, even if I can't speak to the company itself. At the scale of a few hundred to a few thousand documents, especially short documents, there are a few out of the box methods that can yield reasonable results, whether it be embedding clustering o…

Why is it hard? Ultimately you take whatever your signal is and send it to some relatively cheap LLM.

How is it easier to sign up and manage a different service, implement a different API, etc.

And from the company side the fatal flaw is that these types of tools rely upon 1% of their users having huge spend. Nobody is going to be a huge spender here because it's easier to hand roll than navigate procurement on this (not to mention impossible to justify the spend, additional security/privacy risk, etc.)

It feels approximately impossible for this company to have large accounts.

Re: Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

#47

Why... why do companies keep taking every tiny feature and trying to productize it? In the tradition of boring software, even before LLMs it was much simpler to just use your existing tools and hand-roll. With LLMs I cannot fathom reaching for a product for something small like this.

i agree w you for smaller teams tbh. if you have a few hundred convos and someone can maintain scripts/prompts, hand-rolling is probably fine.

it becomes less tiny when it’s 10k+ msgs/week, long voice calls, metadata, changing clusters, retention/redaction, and the team wants this continuously without maintaining another internal tool.

but i'd still wanna know how you'd do this with existing tools before llms?

Re: Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

#48
post #30

I see a fair number of comments here advocating for either codex to hand-roll this themselves, or to simply punt to SQL. I do want to advocate for the difficulty of the problem, even if I can't speak to the company itself. At the scale of a few hundred to a few thousand documents, especially short documents, there are a few out of the box methods that can yield reasonable results, whether it be embedding clustering o…

Why is it hard? Ultimately you take whatever your signal is and send it to some relatively cheap LLM. How is it easier to sign up and manage a different service, implement a different API, etc. And from the company side the fatal flaw is that these types of tools rely upon 1% of their users having huge spend. Nobody is going to be a huge spender here because it's easier to hand roll than navigate procurement on this…

it gets hard when you need this continuously across lots of chats/calls, with metadata, changing clusters, going deeper into a user journey, etc. the LLM call is just one part of it lol

we're keeping it useful every week, finding out insights that the teams can extract value out of, work with them to understand users better.

the procurement what we've seen is v similar to how one would have for any analytics product? and we're selling this to companies when/once it becomes someone's job to do this

Re: Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

#49

I built an in-house version of this a couple of years ago for where I was working. My concern would be that by excluding observability, you might end up creating a really selective dataset, whose conclusions you're then asking companies to take seriously when allocating resources to different possible roadmaps. My guess would be that agent logs would highlight obvious feature requests and bugs for smaller companies -…

i like these kinds of critiques, we don’t think conversation logs or analysis on top of it is alone enough to replace observability or evals. imo they answer diff questions for diff use-cases. we're betting that there is a TONNN of product signal buried in conversations that observability misses, esp around like raging, writing in all caps, repeated prompts, frustration loops, and subtle hidden feature demand. thats…

There's definitely a tonne of signal in those, and it's a critique made from a place of strong support of your basic thesis. There's always been a tonne of signal in traditional customer support requests that goes un-used by most orgs, especially b2c orgs.

In case it's helpful: I always explained it to people I was training like this: All lean product theory comes from listening to the workers actually assembling the parts at Toyota.

Now, most digital products - whether the UI is graphical or linguistic - require a customer to work on an assembly line themselves. An onboarding flow is an assembly line and the user has tasks. Those users complain to agents (whether human or LLM) about their task on the assembly line. The purest implementation of lean philosophy would start with modelling these messages and conversations before it did anything else.

If I were you, I'd build a CRM. Intercom and its ilk charge ridiculous money for functionality that the people using it despise. The existing products in the space optimise for 'serve customers quickly' (increasingly irrelevant with LLMs) and not 'learning from your customers' (increasingly relevant as humans talk to customers less day-to-day). They are horrible to try to integrate into an established product development cycle (I've tried).

I think this makes the proposition easier to comprehend to a customer, the value-add more obvious, and allows you to undercut on pricing, rather than giving people a new bill for something they don't know if they need. The MVP of a CRM is also perhaps easier to build than it might seem initially. "Serve customers faster, cheaper, and learn from them in a highly configurable & meaningfully better way, giving your product iteration an advantage over your competitors". Building a CRM, crucially, allows you oversight of much more of the data - which then enables significantly more meaningful discovery.

This is the unsolved half of the coding agent space: what to actually build, what order to build it in, and why. It's really solvable from your starting point, and is potentially just as important/disruptive as the coding agent has been thus far - especially now that we suddenly have more lines of code than we know what to do with.

I'll shut up now - it's a fascinating space to me, so it's easy to get carried away about! Always happy to talk about stuff like this via email (in my profile) on the off-chance any of the above was useful, though :-)

Re: Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

#50
post #30

I see a fair number of comments here advocating for either codex to hand-roll this themselves, or to simply punt to SQL. I do want to advocate for the difficulty of the problem, even if I can't speak to the company itself. At the scale of a few hundred to a few thousand documents, especially short documents, there are a few out of the box methods that can yield reasonable results, whether it be embedding clustering o…

IMHO this would make more sense if provided as part of a larger "platform" like Langfuse/Langsmith/etc. Otherwise you just end with a dozen SaaS products for highly specific use cases which might not scale that well.

Realistically do you also need to have this live with a fancy? i.e. a custom solution maybe even Jupiter notebooks initially might be sufficient. It's not like 100k messages is a very large dataset. It's not trivial to make a generic solution that fits every use cases (besides of basic customer chatbots) to get actual value for more agentic products.

Post reply on HN