Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI chatbots are banned from our docs for now

mux.com

Post reply on HN
Print view View original

11–18 of 18 posts

Re: AI chatbots are banned from our docs for now

#11

Post by dardarbinks »

dcre wrote: post #5

The lack of detail here makes this post pretty useless, though I guess I’m not surprised generic docs bots aren’t that great. Without knowing any more detail than “We got in touch with a few docs bot services and set up demos that were trained on our docs and blog posts.” it is hard to generalize to RAG + chat in general. I’ve had very good results with a custom setup that uses Claude Haiku to narrow down the set of…

I don't think I mean to indict RAG + chat in general! I think it's totally possible that, if we put more work in, we'd get a great bot out.

But the bar is so, so high though. It's gotta be a truly great bot for us to not be scared of misleading our new users. And I'm still worried that "truly great" is going to take a LOT of work.

And for now, that's the problem. We're still a startup with limited resources. This tool isn't ready for us because we don't have the bandwidth to put the work in.

I can't wait til that bar drops, though. GPT 4o is a really solid step in that direction.

Re: AI chatbots are banned from our docs for now

#12

Post by dardarbinks »

kgeist wrote: post #6

A few days ago I prototyped an AI chatbot which has access to our product's documentation and so far whatever I throw at it, it answers pretty well, without hallucination. It uses GPT4o-mini and OpenSearch for hybrid search (with custom parsing and indexing). After answering a user's question, it also links to the articles where it found the information. My conjectures are: 1) Their bot had bad retrieval. 2) Their bo…

I hope it's not 4, but if it is, let me know!

Re: AI chatbots are banned from our docs for now

#14

Post by dcre »

dardarbinks wrote: post #11
dcre wrote: post #5

The lack of detail here makes this post pretty useless, though I guess I’m not surprised generic docs bots aren’t that great. Without knowing any more detail than “We got in touch with a few docs bot services and set up demos that were trained on our docs and blog posts.” it is hard to generalize to RAG + chat in general. I’ve had very good results with a custom setup that uses Claude Haiku to narrow down the set of…

I don't think I mean to indict RAG + chat in general! I think it's totally possible that, if we put more work in, we'd get a great bot out. But the bar is so, so high though. It's gotta be a truly great bot for us to not be scared of misleading our new users. And I'm still worried that "truly great" is going to take a LOT of work. And for now, that's the problem. We're still a startup with limited resources. This too…

That much I will concede. I said we’ve had good results, but we’ve still been a bit scared to roll it out, more for potential cost and polish reasons than baseline quality, but of course I’m still worried about it saying something wrong.

Re: AI chatbots are banned from our docs for now

#15

Post by dardarbinks »

dardarbinks wrote: post #11
dcre wrote: post #5

The lack of detail here makes this post pretty useless, though I guess I’m not surprised generic docs bots aren’t that great. Without knowing any more detail than “We got in touch with a few docs bot services and set up demos that were trained on our docs and blog posts.” it is hard to generalize to RAG + chat in general. I’ve had very good results with a custom setup that uses Claude Haiku to narrow down the set of…

I don't think I mean to indict RAG + chat in general! I think it's totally possible that, if we put more work in, we'd get a great bot out. But the bar is so, so high though. It's gotta be a truly great bot for us to not be scared of misleading our new users. And I'm still worried that "truly great" is going to take a LOT of work. And for now, that's the problem. We're still a startup with limited resources. This too…

Oh yeah, and I was worried about the "supervised/unsupervised" comment you made.

I'm not talking about supervised training. I think I mean to say that the OUTPUT is supervised/unsupervised. Like, I'm an experienced programmer, so I can supervise the output of Copilot, unlike our unexperienced docs users.

That's on me for not making that train of thought clear enough, and unfortunately choosing a term that's already in use by the AI/ML industry.

Added a footnote to clarify

Re: AI chatbots are banned from our docs for now

#16

Post by sholladay »

I think Discord is where these bots belong. For starters, it’s a context where people expect to chat. People are also accustomed to interacting with bots there, even before the rise of generative AI. And most importantly, unlike when it’s embedded on a website, in Discord the bot can be supervised, corrected by members of the community. You could probably even set up a system where the reactions of certain users, e.g. the project’s maintainers, get fed to the model as training data. I think this could work really well.

One obvious downside is that people may be more reluctant to ask embarrassing questions in public. Though, you could allow DMs to the bot to help with that.

Re: AI chatbots are banned from our docs for now

#17

Post by mananaysiempre »

sholladay wrote: post #16

I think Discord is where these bots belong. For starters, it’s a context where people expect to chat. People are also accustomed to interacting with bots there, even before the rise of generative AI. And most importantly, unlike when it’s embedded on a website, in Discord the bot can be supervised, corrected by members of the community. You could probably even set up a system where the reactions of certain users, e.g…

> in Discord the bot can be supervised, corrected by members of the community.

I have volunteered to answer questions on IRC, sometimes ones that require quite a bit of research. But when I do that, there’s a human on the other end that can learn and at least move in the direction of not having that problem again. I don’t think I’ll ever spend my time correcting the mistakes of a bot that will just as confidently make them time and time again.

Re: AI chatbots are banned from our docs for now

#18

Post by mmcclure »

stainablesteel wrote: post #2

did the ai make a mistake? nah, the codebase it wrong

LLMs definitely make "mistakes". It's well-documented by both users and the providers themselves. Even if 5-10% of questions get a hallucination that sends someone down a totally wrong path, that's too much. It's a really high bar, to be clear, but an important one imo.
Post reply on HN