Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

giskard.ai

Post reply on HN
Print view View original

41–50 of 85 posts

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#41

Post by pornel »

crop_rotation wrote: post #23

What does it even mean to detect hallucinations. The AI doesn't say something trivially false. While using GPT4 I have observed that it lies on simple things I didn't expect it to, while complex things it does very well on. TLDR: It lies on fact based information which is mentioned in very very few places on the internet and not repeated too much. Short of having a human with the context, how do you even detect it. E…

If the hallucinations are random, how about generating multiple answers, and then checking whether they all agree on key points?

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#43

Post by QuercusMax »

crop_rotation wrote: post #34
QuercusMax wrote: post #28

Earlier quoted context omitted.

But isn't that half the reason people are so excited about this stuff - that you can ask it to make up an episode and it does a plausible job.

That is beside the point. My point is that detecting hallucinations seems like a very very hard problem. The utility of it is there and has nothing to do with making up episodes instead of quoting the current one. Like you can ask it to write new episodes with specific settings and specific constraints. Hallucination is not the value add. Nobody is excited because it hallucinates. People are excited despite it since…

But hallucinations are exactly the same thing as asking it to write a spec script.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#44

Post by henri18 »

crop_rotation wrote: post #23

What does it even mean to detect hallucinations. The AI doesn't say something trivially false. While using GPT4 I have observed that it lies on simple things I didn't expect it to, while complex things it does very well on. TLDR: It lies on fact based information which is mentioned in very very few places on the internet and not repeated too much. Short of having a human with the context, how do you even detect it. E…

There are many ways to detect hallucinations. Basically, either you have the ground truth answers in external database, in that case you compare to ground truths. Or you don’t have the ground truth. In that case, you need to do metamorphic testing. See this article on it: https://www.giskard.ai/knowledge/how-to-test-ml-models-4-met...

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#45

Post by thfuran »

QuercusMax wrote: post #28
crop_rotation wrote: post #23

What does it even mean to detect hallucinations. The AI doesn't say something trivially false. While using GPT4 I have observed that it lies on simple things I didn't expect it to, while complex things it does very well on. TLDR: It lies on fact based information which is mentioned in very very few places on the internet and not repeated too much. Short of having a human with the context, how do you even detect it. E…

But isn't that half the reason people are so excited about this stuff - that you can ask it to make up an episode and it does a plausible job.

But deliberately requesting and receiving content generation is altogether different from requesting a factual answer and receiving plausible-seeming nonsense. Or at least, it's different to the person asking; it's the same thing as far as the model is concerned.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#46

Post by taneq »

sebzim4500 wrote: post #6

Seems too good to be true, and I don't understand what it even means for an LLM to be unbiased.

It doesn't. Mean anything, I mean. Language isn't well defined, so its accuracy is also undefined and a non-uniform deviation from that accuracy is super undefined.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#47

Post by taneq »

thfuran wrote: post #42
sebzim4500 wrote: post #6

Seems too good to be true, and I don't understand what it even means for an LLM to be unbiased.

One with only weights and no biases in the ANN is unbiased.

I feel like this went underappreciated. [b] = [0] indeed.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#48

Post by alexcombessie »

amitport wrote: post #33
henri18 wrote: post #25

Earlier quoted context omitted.

It's good to have third parties (apart from Open AI) that assess the quality of Open AI results. It's the way audits work, it has to be independent... Also, third parties are essential to compare the results from ChatGPT with the results of other LLMs. These are important checks to assess the robustness of OpenAI results!

He didn't say it's not important. He is just pointing out that black-box third party verification is not worth much when you can't independently verify the verifiers.

Definitely agree that black boxes are the problem & that one needs to be able to verify the verifiers - FYI that's why Giskard is open-source and that we build in the open. https://www.giskard.ai/knowledge/giskard-log-1-going-open-so...

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#49

Post by crop_rotation »

henri18 wrote: post #44
crop_rotation wrote: post #23

What does it even mean to detect hallucinations. The AI doesn't say something trivially false. While using GPT4 I have observed that it lies on simple things I didn't expect it to, while complex things it does very well on. TLDR: It lies on fact based information which is mentioned in very very few places on the internet and not repeated too much. Short of having a human with the context, how do you even detect it. E…

There are many ways to detect hallucinations. Basically, either you have the ground truth answers in external database, in that case you compare to ground truths. Or you don’t have the ground truth. In that case, you need to do metamorphic testing. See this article on it: https://www.giskard.ai/knowledge/how-to-test-ml-models-4-met...

But GPT4 doesn't hallucinate on things which are popular enough to be replicated enough times on the web as knowledge. It hallucinates on things which are very less likely to be repeated many times. That rules out an external database with true answers. Unless the external database is supposed to contain all info queryable in all ways, in which case the database is just a better version of GPT-X.

The metamorphic testing approach is interesting and might work.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#50

Post by krona »

henri18 wrote: post #44
crop_rotation wrote: post #23

What does it even mean to detect hallucinations. The AI doesn't say something trivially false. While using GPT4 I have observed that it lies on simple things I didn't expect it to, while complex things it does very well on. TLDR: It lies on fact based information which is mentioned in very very few places on the internet and not repeated too much. Short of having a human with the context, how do you even detect it. E…

There are many ways to detect hallucinations. Basically, either you have the ground truth answers in external database, in that case you compare to ground truths. Or you don’t have the ground truth. In that case, you need to do metamorphic testing. See this article on it: https://www.giskard.ai/knowledge/how-to-test-ml-models-4-met...

Metamorphic testing seems to try to map an output of a model to a ground truth, which I guess is great if you have a database of all the known truths in the universe.
Post reply on HN