Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

giskard.ai

Post reply on HN
Print view View original

21–30 of 85 posts

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#21

Post by burnished »

bjt2n3904 wrote: post #9
FeepingCreature wrote: post #8

Earlier quoted context omitted.

It's impossible to answer this without getting political. Instead, let's just say every previous generation has been critically wrong about some things. Statistically, we're unlikely to be the outlier.

dang's put me on notice, so I'm walking on eggshells here. But truth isn't political. As long as we think it is, we will continue to follow the descent into madness. Truth is just that: truth. The only reason we think truth is political, is because our chosen leaders so heavily depend on lies that the truth would destroy their reign.

Eeehhh? Im not sure truth* exists, but there are things that we accept as true and things that are so fundamental that it doesn't occur to us to question them - these things are inherently political. Just to be clear - not using that word to refer to the specific species of polarized discourse that we got in the states, talking about the nature of power and the human condition.

Curious what you consider to be true though? I'm coming at it from the perspective that even in physics where we can isolate so nicely we still aren't divining any truths, just making models with increasing explanatory powers.

Personally, I've been reaching more towards 'shared values' than 'truth', this is likely the pedant in me but truth doesn't feel tractable whereas shared values feels like it has less baggage?

*pretty sure lies exist though

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#22

Post by bluescrn »

bjt2n3904 wrote: post #4

This is like saying, "I've developed a new compass for a deep space probe to help it find North!" Our society is actively declaring that falsehoods are truth, and should be celebrated. We're hallucinating ourselves. All this software does is make sure LLMs hallucinate with us.

Before long, we could end up with left-leaning and right-leaning AIs autonomously fighting the 'culture war' over social media, much more advanced than simple bots spamming copy+paste comments.

Combined with ever-improving ways to fake video and voices, things could get even uglier than they've been over the last few years.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#23

Post by crop_rotation »

What does it even mean to detect hallucinations. The AI doesn't say something trivially false. While using GPT4 I have observed that it lies on simple things I didn't expect it to, while complex things it does very well on.

TLDR: It lies on fact based information which is mentioned in very very few places on the internet and not repeated too much. Short of having a human with the context, how do you even detect it.

Example: Ask it to describe a "Will and Grace" episode with some guest appearance. It will always make up everything including the episode number and the plot, and the plot seems very believable. If you have not watched and can't find a summary online, it is hard to say that it is a lie.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#24

Post by becquerel »

plagiarist wrote: post #7
bjt2n3904 wrote: post #4

This is like saying, "I've developed a new compass for a deep space probe to help it find North!" Our society is actively declaring that falsehoods are truth, and should be celebrated. We're hallucinating ourselves. All this software does is make sure LLMs hallucinate with us.

Such as?

Generally when people talk like this nowadays they mean trans people, or the LGBT community in general. Sometimes Jews, though those types don't say that part out loud on HN too often.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#25

Post by henri18 »

fatso784 wrote: post #16

Looks a bit like snakeoil to me. A lot of companies now spinning up simple demos with opaque backends, making huge claims they’ve solved X hard problem for/with AI, then saying “trust us” and “join our waitlist” without hard details or facts to show for it. If you could detect hallucinations/biases etc that easily, don’t you think OpenAI would’ve worked on something like this?

It's good to have third parties (apart from Open AI) that assess the quality of Open AI results. It's the way audits work, it has to be independent... Also, third parties are essential to compare the results from ChatGPT with the results of other LLMs. These are important checks to assess the robustness of OpenAI results!

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#26

Post by bjt2n3904 »

Ialdaboth wrote: post #13
bjt2n3904 wrote: post #4

This is like saying, "I've developed a new compass for a deep space probe to help it find North!" Our society is actively declaring that falsehoods are truth, and should be celebrated. We're hallucinating ourselves. All this software does is make sure LLMs hallucinate with us.

I'm pretty interested and fascinated in all the efforts that go into making sure it lies as convincingly as us.

I firmly reject that GPT is sentient, or can even tell truth. You're no better off asking it questions about society than asking a magic eight ball.

And yet there's something sinister about twisting an evolutionary model into a reeducation camp.

LLMs are not human, but strikingly similar. If we have no qualms about how we treat it, then what will we do to real people?

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#27

Post by Googleton »

anonzzzies wrote: post #3

How does this work? Does anyone know? And for a large swats of things, how can it possibly work? It’s not possible to say if or if not it is hallucinating code for almost all code and apis, for instance. And I see similar issues with many fields outside pure facts. With privacy issues as well.

We are a team of engineers & researchers on AI Alignment & Safety, we're investigating multiple methods, including metamorphic testing, human feedback, benchmarks with external data sources, and LLM explainability methods.

Currently, fact checking works on straight facts. It does a Google Search and uses LLMs to shorten it. Once it has the short version, it will compare the short results with the answer provided by ChatGPT itself. Premium tiers would get better fact checking sources than just google. We're investigating various data sources and comparison methods.

Note that fact checking / hallucinations is just one of the types of satety issues we'd like to tackle. Many of these are still open questions in the research community, so we're looking to build and develop the right methods for the rights problems. We also think it's super important to have independent third-party evaluations to make sure these models are safe.

This is a new tool we're building in the open, and we're interested in your feedback to prioritize!

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#28

Post by QuercusMax »

crop_rotation wrote: post #23

What does it even mean to detect hallucinations. The AI doesn't say something trivially false. While using GPT4 I have observed that it lies on simple things I didn't expect it to, while complex things it does very well on. TLDR: It lies on fact based information which is mentioned in very very few places on the internet and not repeated too much. Short of having a human with the context, how do you even detect it. E…

But isn't that half the reason people are so excited about this stuff - that you can ask it to make up an episode and it does a plausible job.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#30

Post by jsheard »

henri18 wrote: post #25
fatso784 wrote: post #16

Looks a bit like snakeoil to me. A lot of companies now spinning up simple demos with opaque backends, making huge claims they’ve solved X hard problem for/with AI, then saying “trust us” and “join our waitlist” without hard details or facts to show for it. If you could detect hallucinations/biases etc that easily, don’t you think OpenAI would’ve worked on something like this?

It's good to have third parties (apart from Open AI) that assess the quality of Open AI results. It's the way audits work, it has to be independent... Also, third parties are essential to compare the results from ChatGPT with the results of other LLMs. These are important checks to assess the robustness of OpenAI results!

I can't help but notice your accounts only activity before this post was praising another giskard.ai submission a few months ago. Anything you'd like to disclose?
Post reply on HN