Would be great if it pinged me on slack or whatsapp. I wouldn't notice if it simply paused waiting for the MCP call to return
Easy enough to do with smolagents and fastmcp, its 20 lines of code.
Show HN: Ask-human-mcp – zero-config human-in-loop hatch to stop hallucinations
41–50 of 65 posts
Re: Show HN: Ask-human-mcp – zero-config human-in-loop hatch to stop hallucinations
#42If the model responds with an obvious incorrect answer or hallucination, start over. Rephrase your input. Consider what output you are actually after... Adding to original shit output wont help you.
Re: Show HN: Ask-human-mcp – zero-config human-in-loop hatch to stop hallucinations
#43Why wouldn't a rag-enabled ai be faster and better then humans at answering these documentation-grounded questions ?
Re: Show HN: Ask-human-mcp – zero-config human-in-loop hatch to stop hallucinations
#44- someone sets up an “ask human as a service mcp” - demand quickly outstrips offer of humans willing to help bots - someone else hooks up AI to the “ask human saas” - we now have a full loop of machines asking machines
Finally, the "AI" turns out to be 700 Indians. We now have the full loop of humans asking machines asking humans pretending to be machines. Civilisation collapses
AI stands for Actual Indians.
Re: Show HN: Ask-human-mcp – zero-config human-in-loop hatch to stop hallucinations
#45Earlier quoted context omitted.
I'll try to give you credit for more than dismissing my question off-hand... Yes, it may not need to know with perfect certainty when it's unsure or stuck, but even to meet a lower bar of usefulness, it'll need at least an approximate means of determining that its knowledge is inadequate. To purport to help with the hallucination problem requires no less. To make the issue a bit more clear, here are some candidate co…
You are trying to control a system that is inherently chaotic. You can probably get some where by indeed running a task 1000 times and looking for outliers in the execution time or token count. But that is of minimal use and anything more advanced than that is akin to water divining.
The system is only nondeterministic (and a model of nondeterminism at that) when it's emitting tokens. It (the system) becomes completely deterministic when it calls a tool and a result is returned from the tool.
This is little different than how I wrote this. Now it is deterministic, when I hit reply.
Re: Show HN: Ask-human-mcp – zero-config human-in-loop hatch to stop hallucinations
#46What sort of prompt are you using for this?
The prompt is (mostly) built using the tool loads in the MCP server. In Python, the @mcp.tool() decorators provide the context of tool to the prompt, which is then submitted (I believe) with each call to the LLM.
Re: Show HN: Ask-human-mcp – zero-config human-in-loop hatch to stop hallucinations
#47The same technique can be had by creating a "universal MCP tool" for the LLM to use if it thinks the existing tools aren't up to the job. The MCP language calls these "proxies".
Re: Show HN: Ask-human-mcp – zero-config human-in-loop hatch to stop hallucinations
#48Re: Show HN: Ask-human-mcp – zero-config human-in-loop hatch to stop hallucinations
#49Earlier quoted context omitted.
reasoning models know when they are close to hallucinating because they are lacking context or understanding and know that they could solve this with a question. this is a streamlined implementation of a interanlly scrapped together tool that i decided to open-source for people to either us or build off of.
> reasoning models know when they are close to hallucinating because they are lacking context or understanding and know that they could solve this with a question You've just described AGI. If this were possible you could create an MCP server that has a continually updated list of FAQ of everything that the model doesn't know. Over time it would learn everything.
Unless there is as yet insufficient data for meaningful answer.
Re: Show HN: Ask-human-mcp – zero-config human-in-loop hatch to stop hallucinations
#50Earlier quoted context omitted.
>If you are answering these questions yourself, why not just add something like this to your cursor rules? What you are asking for is AGI. We still need human in the loop for now.
What I'm describing is a human in the loop. It's just a different UX, one that is easier to use and closer to what the model is trained to use.
Human in the loop means despite your best efforts at initial prompting (which is what rules are), there will always be the need to say "no, that's wrong, now do this instead". Expecting to be able to write enough rules for the model to work fully autonomously through your problem is indeed wishing for AGI.