Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

giskard.ai

Post reply on HN
Print view View original

31–40 of 85 posts

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#31

Post by Veen »

bjt2n3904 wrote: post #9
FeepingCreature wrote: post #8

Earlier quoted context omitted.

It's impossible to answer this without getting political. Instead, let's just say every previous generation has been critically wrong about some things. Statistically, we're unlikely to be the outlier.

dang's put me on notice, so I'm walking on eggshells here. But truth isn't political. As long as we think it is, we will continue to follow the descent into madness. Truth is just that: truth. The only reason we think truth is political, is because our chosen leaders so heavily depend on lies that the truth would destroy their reign.

Some of the ugliest episodes in human history were caused by people who believed their political positions were not political positions, but unarguable statements of the True and Good.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#33

Post by amitport »

henri18 wrote: post #25
fatso784 wrote: post #16

Looks a bit like snakeoil to me. A lot of companies now spinning up simple demos with opaque backends, making huge claims they’ve solved X hard problem for/with AI, then saying “trust us” and “join our waitlist” without hard details or facts to show for it. If you could detect hallucinations/biases etc that easily, don’t you think OpenAI would’ve worked on something like this?

It's good to have third parties (apart from Open AI) that assess the quality of Open AI results. It's the way audits work, it has to be independent... Also, third parties are essential to compare the results from ChatGPT with the results of other LLMs. These are important checks to assess the robustness of OpenAI results!

He didn't say it's not important. He is just pointing out that black-box third party verification is not worth much when you can't independently verify the verifiers.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#34

Post by crop_rotation »

QuercusMax wrote: post #28
crop_rotation wrote: post #23

What does it even mean to detect hallucinations. The AI doesn't say something trivially false. While using GPT4 I have observed that it lies on simple things I didn't expect it to, while complex things it does very well on. TLDR: It lies on fact based information which is mentioned in very very few places on the internet and not repeated too much. Short of having a human with the context, how do you even detect it. E…

But isn't that half the reason people are so excited about this stuff - that you can ask it to make up an episode and it does a plausible job.

That is beside the point. My point is that detecting hallucinations seems like a very very hard problem.

The utility of it is there and has nothing to do with making up episodes instead of quoting the current one. Like you can ask it to write new episodes with specific settings and specific constraints. Hallucination is not the value add. Nobody is excited because it hallucinates. People are excited despite it since the other value add is too much.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#35

Post by chaxor »

burnished wrote: post #21
bjt2n3904 wrote: post #9

Earlier quoted context omitted.

dang's put me on notice, so I'm walking on eggshells here. But truth isn't political. As long as we think it is, we will continue to follow the descent into madness. Truth is just that: truth. The only reason we think truth is political, is because our chosen leaders so heavily depend on lies that the truth would destroy their reign.

Eeehhh? Im not sure truth* exists, but there are things that we accept as true and things that are so fundamental that it doesn't occur to us to question them - these things are inherently political. Just to be clear - not using that word to refer to the specific species of polarized discourse that we got in the states, talking about the nature of power and the human condition. Curious what you consider to be true th…

Does shared values here just mean definitions? Such as the number of carbon atoms in a mole, 5+9 in base 10, the average number of protons in a carbon atom is a specific value, and leptons exist?

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#36

Post by mirker »

henri18 wrote: post #25
fatso784 wrote: post #16

Looks a bit like snakeoil to me. A lot of companies now spinning up simple demos with opaque backends, making huge claims they’ve solved X hard problem for/with AI, then saying “trust us” and “join our waitlist” without hard details or facts to show for it. If you could detect hallucinations/biases etc that easily, don’t you think OpenAI would’ve worked on something like this?

It's good to have third parties (apart from Open AI) that assess the quality of Open AI results. It's the way audits work, it has to be independent... Also, third parties are essential to compare the results from ChatGPT with the results of other LLMs. These are important checks to assess the robustness of OpenAI results!

The OPs point is that it’s likely impossible to do what is claimed here in general. Imagine the LLM says something like Fermat’s Last Theorem. To verify it, you’d have to either 1) have a proof assistant powerful enough to construct a proof 2) use a second ML model to guess truthfulness. The former is technically challenging and the latter is another model, with its own biases and factual inconsistencies.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#37

Post by muxl »

SamoyedFurFluff wrote: post #17
bjt2n3904 wrote: post #9

Earlier quoted context omitted.

dang's put me on notice, so I'm walking on eggshells here. But truth isn't political. As long as we think it is, we will continue to follow the descent into madness. Truth is just that: truth. The only reason we think truth is political, is because our chosen leaders so heavily depend on lies that the truth would destroy their reign.

I’m a little confused on the statement “truth isn’t political”. This kind of goes against what I understand politics to be, which is the negotiation of a broader societal trend, which doesn’t itself have to do with whether or not the societal trend has a factual basis. The truth may be that cigarettes cause cancer, but the politics are obviously that acknowledging this would encourage society to implement top down po…

If I'm understanding the parent comment correctly: a fact may have political implications but it doesn't depend on politics. In other words reality is independent of our interpretation of it (i.e. philosophical realism). The rub of course being that coming to know facts about most things is a highly social process filtered through interpretation and biases. Everything can be political if it needs to be decided upon by a group.

EDIT: I have avoided using "truth" here because it's a more general term than "fact" which has the connotation of being in reference to something concrete.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#38

Post by Turing_Machine »

sebzim4500 wrote: post #6

Seems too good to be true, and I don't understand what it even means for an LLM to be unbiased.

"Unbiased" almost always means "has biases that are similar to mine". I can't think of very many exceptions to that, frankly.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#39

Post by mountainb »

bjt2n3904 wrote: post #4

This is like saying, "I've developed a new compass for a deep space probe to help it find North!" Our society is actively declaring that falsehoods are truth, and should be celebrated. We're hallucinating ourselves. All this software does is make sure LLMs hallucinate with us.

Well I think that its hallucinations are a good demonstration that none of the post-truth subjectivist philosophies were ever things that many people took very seriously. ChatGPT is the real thing as it relates to extreme relativism: it really cannot tell the difference between true and false and doesn't really care, either. By contrast, the apostles of post-truth were really only trolling for a response... no one really lives by its principles because it's so impractical and disastrous. To really believe in post-truth, you must perceive no difference between raisin bran with cyanide and raisin bran.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#40

Post by henri18 »

sebzim4500 wrote: post #6

Seems too good to be true, and I don't understand what it even means for an LLM to be unbiased.

There are many bias detectors developed in the research world. To have a deeper look, you can look at this paper: https://arxiv.org/abs/2208.05777
Post reply on HN