11–20 of 48 posts
I think it's a good idea and someone will see success here, unfortunately it's far too slow to be useful right now (HN effect?)
Can the chat be handed off to a human seamlessly in the event the bot can't answer the question?
Ideally we could be the UI layer for intercom, zendesk, etc. We already do that for docs search / exploration.
Tested out the landing page chatbot and got one “We couldnt find an answer to your question” and another request timeout. Not confident at all using this with any product.
I've wanted a product like this since I first encountered chatGPT. How do you handle curation? Meaning... if the model picks up some out of date info or misinterprets it, and a human admin notices that and wants to mark something as out of date or wrong, can they? I see this as similar to the way I can correct ChatGPT over the course of a chat session, and it will remember the corrections.
Also thinking about ways to ensure the bot answers common questions correctly while still being able to personalize responses. Working on something called "answer shaping" where an admin can write out a response and tag with the question it responds to. Then the bot would first check to see if the human question matches a cached question, and if so would prioritize using info in the cached answer in its response. Seems like this can give the bot freedom to personalize the answer but make sure it includes the right stuff.
Congrats on the launch! How do you approach handling sensitive data that might come from a CMS in an AI context? Both: 1. Sensitive data being surfaced accidentally during regular conversation and 2. Malicious actors using prompt injection or similar techniques?
Nothing revolutionary to report on the prompt injection stuff. Most people using HH are using it for public documentation so there really isn't any info in the source content that couldn't be surfaced in an answer.
Tested out the landing page chatbot and got one “We couldnt find an answer to your question” and another request timeout. Not confident at all using this with any product.
Oops looking into it. Do you remember which bot you used and what question you asked?
I've wanted a product like this since I first encountered chatGPT. How do you handle curation? Meaning... if the model picks up some out of date info or misinterprets it, and a human admin notices that and wants to mark something as out of date or wrong, can they? I see this as similar to the way I can correct ChatGPT over the course of a chat session, and it will remember the corrections.
If we're doing our job right then HH should only be answering based on its source content (and not background knowledge). So bad answers coming from incorrect source content would need to be corrected in the source content. Citations should help with this, e.g. as a human admit, review answer -> notice it's bad -> click citation -> locate incorrect part -> change it. Also thinking about ways to ensure the bot answers…
I understand why you want the flow to work this way (single source of truth, fix things at the root, simplifies everything), but, respectfully, it is really bad from a UX perspective. Here are the main reasons:
1. Not all admins will have the ability to edit all source pages. Both from a permissions perspective (eg, a zendesk ticket or slack message created by someone else) or a technical ability perspective (eg, you need to edit html and create a PR).
2. People are busy and lazy. If I can see the problem in the answer, notice it's wrong, and correct it right now on the page where I see it, I will. Otherwise, I often won't. Think busy CS agents, developers in the midst of problem solving, etc.
Yes, supporting this workflow makes life harder on you, because it's technically more complex, but it's the way people will want to use this product.
I think it's a good idea and someone will see success here, unfortunately it's far too slow to be useful right now (HN effect?)
HN has slowed it down a bit but speed is definitely not where I want it to be. A lot of that comes from the openai side atm. Just turned on caching for the suggested questions so at least those are extremely fast.
I realize this is just one part of your product, but what if OpenAI goes away (not directly, but effectively)? Is there a contingency plan to move to another LLM? Build something else?