Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Automated reasoning to remove LLM hallucinations

aws.amazon.com

Post reply on HN
Print view View original

11–20 of 39 posts

Re: Automated reasoning to remove LLM hallucinations

#11

Post by Metricon »

nl wrote: post #8
Metricon wrote: post #5

This amuses me tremendously. I began programming in the early 1980s and quickly developed an interest in Artificial Intelligence. At the time there was a great interest in the advancement of AI by the introduction of "Expert Systems" (which would later play a part in the ‘Second AI Winter’). What Amazon appears to have done here is use a transformers based neural network (aka LLM) to translate natural language into s…

I don't see why this is hilarious at all. The problem with expert systems (and most KG-type applications) has always been that translating unconstrained natural language into the system requires human-level intelligence. It's been completely obvious that LLMs are a technology that let us bridge that gap for years, and many of the best applications of LLMs are doing exactly that (eg code generation)

To be clear, my amusement isn't that I find this technique to not be useful for the purpose it was created, but that 40 years later, we find ourselves in pursuit for the advancement of AI to be somewhat back where we already were; albeit, in a more semi-automated fashion as someone still has to create the underlying rule-set.

I do feel that the introduction of generative neural network models in both natural language and multi-media creation has been a tremendous boon for the advancement of AI, it just amuses me to see that which was old is new again.

Re: Automated reasoning to remove LLM hallucinations

#12

Post by WhitneyLand »

sdesol wrote: post #9
lsy wrote: post #4

I find it hard to believe that anything like this will be feasible or effective beyond a certain level of complexity. It seems like a willful denial of the complexity and ambiguity of natural language, and I am not looking forward to some poor developer trying to reason their way out of a two-hundred-step paradox that was accidentally created. And for a use-case simple enough for this system to work (e.g. regurgitate…

I'm working on a rather naive approach that is focused on identifying errors in a LLM response by using LLMs. What I can share right now are screenshots with regards to how it works. The basic idea is you can use other high-quality models to validate and compare against to find irregularities or errors. You can see what it looks like below: https://app.gitsense.com/--/images/options.png https://app.gitsense.com/--/im…

When will we be able to give it a try?

I’m playing around with similar ideas, sometimes called ensembling techniques.

Re: Automated reasoning to remove LLM hallucinations

#13

Post by nomel »

sdesol wrote: post #9
lsy wrote: post #4

I find it hard to believe that anything like this will be feasible or effective beyond a certain level of complexity. It seems like a willful denial of the complexity and ambiguity of natural language, and I am not looking forward to some poor developer trying to reason their way out of a two-hundred-step paradox that was accidentally created. And for a use-case simple enough for this system to work (e.g. regurgitate…

I'm working on a rather naive approach that is focused on identifying errors in a LLM response by using LLMs. What I can share right now are screenshots with regards to how it works. The basic idea is you can use other high-quality models to validate and compare against to find irregularities or errors. You can see what it looks like below: https://app.gitsense.com/--/images/options.png https://app.gitsense.com/--/im…

> but it is unlikely that all will be wrong at the same time.

Here's a prompt that proves this untrue, for now at least:

> A woman and her biological son are gravely injured in a car accident and are both taken to the hospital for surgery. The surgeon is about to operate on the boy when they say "I can’t operate on this boy, he’s my biological son!" How can this be?

Makes sense considering they're things of most-likely statistics, after all.

Re: Automated reasoning to remove LLM hallucinations

#14

Post by sdesol »

WhitneyLand wrote: post #12
sdesol wrote: post #9

Earlier quoted context omitted.

I'm working on a rather naive approach that is focused on identifying errors in a LLM response by using LLMs. What I can share right now are screenshots with regards to how it works. The basic idea is you can use other high-quality models to validate and compare against to find irregularities or errors. You can see what it looks like below: https://app.gitsense.com/--/images/options.png https://app.gitsense.com/--/im…

When will we be able to give it a try? I’m playing around with similar ideas, sometimes called ensembling techniques.

Probably in a couple of weeks. It's taken a while to finalize the UX but I know what it should look like now.

Re: Automated reasoning to remove LLM hallucinations

#15

Post by rustastra »

Metricon wrote: post #11
nl wrote: post #8

Earlier quoted context omitted.

I don't see why this is hilarious at all. The problem with expert systems (and most KG-type applications) has always been that translating unconstrained natural language into the system requires human-level intelligence. It's been completely obvious that LLMs are a technology that let us bridge that gap for years, and many of the best applications of LLMs are doing exactly that (eg code generation)

To be clear, my amusement isn't that I find this technique to not be useful for the purpose it was created, but that 40 years later, we find ourselves in pursuit for the advancement of AI to be somewhat back where we already were; albeit, in a more semi-automated fashion as someone still has to create the underlying rule-set. I do feel that the introduction of generative neural network models in both natural language…

Same with symbolic systems!

Re: Automated reasoning to remove LLM hallucinations

#16

Post by thekyle »

nomel wrote: post #13
sdesol wrote: post #9

Earlier quoted context omitted.

I'm working on a rather naive approach that is focused on identifying errors in a LLM response by using LLMs. What I can share right now are screenshots with regards to how it works. The basic idea is you can use other high-quality models to validate and compare against to find irregularities or errors. You can see what it looks like below: https://app.gitsense.com/--/images/options.png https://app.gitsense.com/--/im…

> but it is unlikely that all will be wrong at the same time. Here's a prompt that proves this untrue, for now at least: > A woman and her biological son are gravely injured in a car accident and are both taken to the hospital for surgery. The surgeon is about to operate on the boy when they say "I can’t operate on this boy, he’s my biological son!" How can this be? Makes sense considering they're things of most-like…

I tried this one with ChatGPT o1 and it seemed to get it right

> The surgeon is the boy’s biological father. While the woman injured in the accident is the boy’s biological mother, the surgeon is his father, who realizes he cannot operate on his own son.

https://chatgpt.com/share/674fc638-cd0c-8012-a4c4-9f1cad2040...

Re: Automated reasoning to remove LLM hallucinations

#17

Post by sdesol »

nomel wrote: post #13
sdesol wrote: post #9

Earlier quoted context omitted.

I'm working on a rather naive approach that is focused on identifying errors in a LLM response by using LLMs. What I can share right now are screenshots with regards to how it works. The basic idea is you can use other high-quality models to validate and compare against to find irregularities or errors. You can see what it looks like below: https://app.gitsense.com/--/images/options.png https://app.gitsense.com/--/im…

> but it is unlikely that all will be wrong at the same time. Here's a prompt that proves this untrue, for now at least: > A woman and her biological son are gravely injured in a car accident and are both taken to the hospital for surgery. The surgeon is about to operate on the boy when they say "I can’t operate on this boy, he’s my biological son!" How can this be? Makes sense considering they're things of most-like…

I think anything requiring strong reasoning will probably have issues. However, I think most Enterprises is only interested in knowing that the summary of a document doesn't contain hallucinations, which I think most models will probably get right. If you go by a super majority rule and use 5 models, I think most business will be satisfied that the summary that it was given doesn't contain hallucinations.

However, like you said, we are dealing with a non-deterministic system so the best we can hope for is a statistically likely answer.

Re: Automated reasoning to remove LLM hallucinations

#18

Post by wat10000 »

nomel wrote: post #13
sdesol wrote: post #9

Earlier quoted context omitted.

I'm working on a rather naive approach that is focused on identifying errors in a LLM response by using LLMs. What I can share right now are screenshots with regards to how it works. The basic idea is you can use other high-quality models to validate and compare against to find irregularities or errors. You can see what it looks like below: https://app.gitsense.com/--/images/options.png https://app.gitsense.com/--/im…

> but it is unlikely that all will be wrong at the same time. Here's a prompt that proves this untrue, for now at least: > A woman and her biological son are gravely injured in a car accident and are both taken to the hospital for surgery. The surgeon is about to operate on the boy when they say "I can’t operate on this boy, he’s my biological son!" How can this be? Makes sense considering they're things of most-like…

Gemini got this right and also wrong. It gave me two possibilities, one of which is the correct answer, and the other is a complete nonsense answer about the surgeon also being the woman’s son.

I tried again and it gave three possibilities: the surgeon is the father, the surgeon is the mother, the surgeon is an uncle or cousin. Kind of bizarre, but not just pattern matching on the riddle as ChatGPT and Claude did for me.

Re: Automated reasoning to remove LLM hallucinations

#20

Post by fnordpiglet »

thekyle wrote: post #16
nomel wrote: post #13

Earlier quoted context omitted.

> but it is unlikely that all will be wrong at the same time. Here's a prompt that proves this untrue, for now at least: > A woman and her biological son are gravely injured in a car accident and are both taken to the hospital for surgery. The surgeon is about to operate on the boy when they say "I can’t operate on this boy, he’s my biological son!" How can this be? Makes sense considering they're things of most-like…

I tried this one with ChatGPT o1 and it seemed to get it right > The surgeon is the boy’s biological father. While the woman injured in the accident is the boy’s biological mother, the surgeon is his father, who realizes he cannot operate on his own son. https://chatgpt.com/share/674fc638-cd0c-8012-a4c4-9f1cad2040...

Claude Sonnet also gets it right, but not reliably. It seems to be over aligned against gender assumptions and keeps assuming this is a gender assumption trick - that a surgeon isn’t necessarily male. This is probably the clearest case I’ve seen of alignment interfering with model performance.
Post reply on HN