Live data from Hacker News

Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

agnost.ai

51–60 of 63 posts

Re: Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

#51

Earlier quoted context omitted.

Why is it hard? Ultimately you take whatever your signal is and send it to some relatively cheap LLM. How is it easier to sign up and manage a different service, implement a different API, etc. And from the company side the fatal flaw is that these types of tools rely upon 1% of their users having huge spend. Nobody is going to be a huge spender here because it's easier to hand roll than navigate procurement on this…

it gets hard when you need this continuously across lots of chats/calls, with metadata, changing clusters, going deeper into a user journey, etc. the LLM call is just one part of it lol we're keeping it useful every week, finding out insights that the teams can extract value out of, work with them to understand users better. the procurement what we've seen is v similar to how one would have for any analytics product?…

If you get this to work once rerunning it weekly seems fairly easy unless you actually need to see the data live and have perfect uptime?

Re: Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

#54
post #51

Earlier quoted context omitted.

it gets hard when you need this continuously across lots of chats/calls, with metadata, changing clusters, going deeper into a user journey, etc. the LLM call is just one part of it lol we're keeping it useful every week, finding out insights that the teams can extract value out of, work with them to understand users better. the procurement what we've seen is v similar to how one would have for any analytics product?…

If you get this to work once rerunning it weekly seems fairly easy unless you actually need to see the data live and have perfect uptime?

a lot of our customers want a daily morning report on slack & flag things instantly rather than to wait for a week so thats why we keep it realtime

Re: Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

#55

Earlier quoted context omitted.

i like these kinds of critiques, we don’t think conversation logs or analysis on top of it is alone enough to replace observability or evals. imo they answer diff questions for diff use-cases. we're betting that there is a TONNN of product signal buried in conversations that observability misses, esp around like raging, writing in all caps, repeated prompts, frustration loops, and subtle hidden feature demand. thats…

There's definitely a tonne of signal in those, and it's a critique made from a place of strong support of your basic thesis. There's always been a tonne of signal in traditional customer support requests that goes un-used by most orgs, especially b2c orgs. In case it's helpful: I always explained it to people I was training like this: All lean product theory comes from listening to the workers actually assembling the…

its fascinating how the toyota example comes up anywhere, its so good!

wdym by modelling the messages and conversations though? i lose you a bit there! for the crm approach, i do think it'll be a problem at some point right now.

the replacing budge is an interesting piece, we did not think of it yet, yes let me dm you on x!

Re: Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

#56
post #50
post #30

I see a fair number of comments here advocating for either codex to hand-roll this themselves, or to simply punt to SQL. I do want to advocate for the difficulty of the problem, even if I can't speak to the company itself. At the scale of a few hundred to a few thousand documents, especially short documents, there are a few out of the box methods that can yield reasonable results, whether it be embedding clustering o…

IMHO this would make more sense if provided as part of a larger "platform" like Langfuse/Langsmith/etc. Otherwise you just end with a dozen SaaS products for highly specific use cases which might not scale that well. Realistically do you also need to have this live with a fancy? i.e. a custom solution maybe even Jupiter notebooks initially might be sufficient. It's not like 100k messages is a very large dataset. It's…

i get the push, most teams come to us after they've done/tired of the claude running analysis thing manually and want a pro-active thing.

we're also targeting conversation first use cases and for them this serves as their everything custoemrs. we obv do not sell the fancy part, idts that sells anymore lol. lot of our queries come from our slack app/mcp.

Re: Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

#57
post #50

Earlier quoted context omitted.

IMHO this would make more sense if provided as part of a larger "platform" like Langfuse/Langsmith/etc. Otherwise you just end with a dozen SaaS products for highly specific use cases which might not scale that well. Realistically do you also need to have this live with a fancy? i.e. a custom solution maybe even Jupiter notebooks initially might be sufficient. It's not like 100k messages is a very large dataset. It's…

i get the push, most teams come to us after they've done/tired of the claude running analysis thing manually and want a pro-active thing. we're also targeting conversation first use cases and for them this serves as their everything custoemrs. we obv do not sell the fancy part, idts that sells anymore lol. lot of our queries come from our slack app/mcp.

[dead]

Re: Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

#58
post #15
post #7

Earlier quoted context omitted.

i mean i would get codex to build everything you just described

Do it then.. the hubris of vibecoders is really something.

To be fair if you have some background in data science (I think the main thing is being able to verify its not showing nonsense) it shouldn’t be too hard to build a bespoke solution.

Re: Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

#59
The distinction between observability and discovery is important. Observability answers "what broke" — discovery answers "what should we build". How you handle sessions that span multiple interactions or context resets? A user might express frustration in message 3 of a conversation but the root cause was a tool failure in message 1 that the agent recovered from poorly!

Re: Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

#60
Congrats on the launch. Question about the data side: conversations tend to contain a lot of PII (names, emails, account details users paste in). When you cluster and surface example messages in the dashboard, is there any redaction step before storage/display, or is that left to the customer to filter before sending through the SDK? Curious how much of SOC 2 scope covers the message content itself vs the infrastructure.
Post reply on HN