31–40 of 63 posts
It’s a cool product and I’m curious to see where you go. We build an MCP factory, where our enterprise customers use our product to build MCPs that their employees use in Claude or Codex. What would be cool for me is if I could use this to surface insights to them, rather than just to our team.
Without using agnost, what are some basic SQL queries I can run on my data to find outliers I'd otherwise be missing? How far can I get with just keywords, common phrases, boring traditional analysis? Depending on what I measure there, when is the right time for me to consider upgrading to something like Agnost/what is a specific example of what it will find that traditional/rigid analytics approaches will miss?
I see a fair number of comments here advocating for either codex to hand-roll this themselves, or to simply punt to SQL. I do want to advocate for the difficulty of the problem, even if I can't speak to the company itself. At the scale of a few hundred to a few thousand documents, especially short documents, there are a few out of the box methods that can yield reasonable results, whether it be embedding clustering o…
model drifting is something a lot of folks do face after 5th/6th turn as per my understanding and it usually the median, how did you tackle it if you have yet?
also yea, thats why we went for a per customer taxonomy than a general one, yeilded better results + easier to improve upon.
I built an in-house version of this a couple of years ago for where I was working. My concern would be that by excluding observability, you might end up creating a really selective dataset, whose conclusions you're then asking companies to take seriously when allocating resources to different possible roadmaps. My guess would be that agent logs would highlight obvious feature requests and bugs for smaller companies -…
we're betting that there is a TONNN of product signal buried in conversations that observability misses, esp around like raging, writing in all caps, repeated prompts, frustration loops, and subtle hidden feature demand. thats also why we use per-customer taxonomies instead of a shared one. evals will still be needed.
the root cause is harder, especially in more mature agents. we're using this more as a discovery layer for evals or even just whats happening kind of things, then letting teams go deep into the actual conversations and decide what to take action upon
Great launch!! There’s a lot of very silly comments of people saying they will vibe code this… errr good luck being the slop version of this startup. :/ It’s a cool product and I’m curious to see where you go. We build an MCP factory, where our enterprise customers use our product to build MCPs that their employees use in Claude or Codex. What would be cool for me is if I could use this to surface insights to them, r…
we havnt figured out yet how to do the b2b2b kinda thing where we surface insights for a multi-tenant sort of approach but i've gotten this now twice in the last hour so happy to chat.
what would you want them to see first though, are these semantic insights like we do today or more around deterministic tool calls/etc metrics?
Great launch!! There’s a lot of very silly comments of people saying they will vibe code this… errr good luck being the slop version of this startup. :/ It’s a cool product and I’m curious to see where you go. We build an MCP factory, where our enterprise customers use our product to build MCPs that their employees use in Claude or Codex. What would be cool for me is if I could use this to surface insights to them, r…
i love this comment bec we started as analytics for mcp servers haha! we then expanded to conversations bec thats where most mcp servers were being used lol. we havnt figured out yet how to do the b2b2b kinda thing where we surface insights for a multi-tenant sort of approach but i've gotten this now twice in the last hour so happy to chat. what would you want them to see first though, are these semantic insights lik…
Earlier quoted context omitted.
i love this comment bec we started as analytics for mcp servers haha! we then expanded to conversations bec thats where most mcp servers were being used lol. we havnt figured out yet how to do the b2b2b kinda thing where we surface insights for a multi-tenant sort of approach but i've gotten this now twice in the last hour so happy to chat. what would you want them to see first though, are these semantic insights lik…
our current prototype of this functionality is fairly basic and surfaces things from the underlying chats like: Users want a "create ticket tool", or "The run SOQL query often fails because it references fields that don't exist. You should provide documentation of the actual fields". Happy to chat if you want to you can find me on bookface as the founder of credal
[flagged]
I see a fair number of comments here advocating for either codex to hand-roll this themselves, or to simply punt to SQL. I do want to advocate for the difficulty of the problem, even if I can't speak to the company itself. At the scale of a few hundred to a few thousand documents, especially short documents, there are a few out of the box methods that can yield reasonable results, whether it be embedding clustering o…
yea, at our volume which we still consider small as we've been able to figure out a way with llms & embeddings, its still fine. + we onboarded a voice ai company with more than 2 hour calls and thats when it was super hard to solve since there were so many elements to consider. model drifting is something a lot of folks do face after 5th/6th turn as per my understanding and it usually the median, how did you tackle i…
Regarding my experience, I have done a fair amount of work in the contact center space with long calls. I used statistical Bayesian approaches which I found to be much more resilient especially on long documents than embeddings/transformers. It also provided a joint modeling foundation for classification with much lower label requirements than BERT or traditional ML.