Live data from Hacker News

SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

giskard.ai

11–20 of 85 posts

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#11

This is like saying, "I've developed a new compass for a deep space probe to help it find North!" Our society is actively declaring that falsehoods are truth, and should be celebrated. We're hallucinating ourselves. All this software does is make sure LLMs hallucinate with us.

Such as?

[flagged]

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#12
These are the kinds of things I can see taking off, for better or worse. I know Adobe's product is worse than Midjourney's, for example, but once the hype meets reality, companies are going to want to be safe when they start using AI formally.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#13

This is like saying, "I've developed a new compass for a deep space probe to help it find North!" Our society is actively declaring that falsehoods are truth, and should be celebrated. We're hallucinating ourselves. All this software does is make sure LLMs hallucinate with us.

I'm pretty interested and fascinated in all the efforts that go into making sure it lies as convincingly as us.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#14

How does this work? Does anyone know? And for a large swats of things, how can it possibly work? It’s not possible to say if or if not it is hallucinating code for almost all code and apis, for instance. And I see similar issues with many fields outside pure facts. With privacy issues as well.

Looking at their “documentation”: https://docs.giskard.ai/start/

It would appear that this is not automated monitoring but more like a second stage of human reinforcement learning or perhaps a classifier. It seems that you create input/output examples and the LLM responses are examined by a secondary system (which I’m guessing is probably NOT an LLM, otherwise it would be vulnerable to attacks) and perhaps force regenerates the LLM response if it doesn’t meet the classification threshold.

At least, that sounds more believable to me than someone claiming they’ve fixed the inherent flaws in LLMs.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#15

This is like saying, "I've developed a new compass for a deep space probe to help it find North!" Our society is actively declaring that falsehoods are truth, and should be celebrated. We're hallucinating ourselves. All this software does is make sure LLMs hallucinate with us.

Such as?

I think its easy to guess what the commenter is obliquely referring to.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#16
Looks a bit like snakeoil to me. A lot of companies now spinning up simple demos with opaque backends, making huge claims they’ve solved X hard problem for/with AI, then saying “trust us” and “join our waitlist” without hard details or facts to show for it. If you could detect hallucinations/biases etc that easily, don’t you think OpenAI would’ve worked on something like this?

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#17

Earlier quoted context omitted.

It's impossible to answer this without getting political. Instead, let's just say every previous generation has been critically wrong about some things. Statistically, we're unlikely to be the outlier.

dang's put me on notice, so I'm walking on eggshells here. But truth isn't political. As long as we think it is, we will continue to follow the descent into madness. Truth is just that: truth. The only reason we think truth is political, is because our chosen leaders so heavily depend on lies that the truth would destroy their reign.

I’m a little confused on the statement “truth isn’t political”. This kind of goes against what I understand politics to be, which is the negotiation of a broader societal trend, which doesn’t itself have to do with whether or not the societal trend has a factual basis. The truth may be that cigarettes cause cancer, but the politics are obviously that acknowledging this would encourage society to implement top down policies to limit cigarette use. In this way, “cigarettes is a carcinogen” is a truth with significant political weight, which is what I understand a political truth to mean. Is my understanding different from yours?

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#19

This is like saying, "I've developed a new compass for a deep space probe to help it find North!" Our society is actively declaring that falsehoods are truth, and should be celebrated. We're hallucinating ourselves. All this software does is make sure LLMs hallucinate with us.

Such as?

People who write with their left hand existing.

Re: SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

#20

Earlier quoted context omitted.

Such as?

It's impossible to answer this without getting political. Instead, let's just say every previous generation has been critically wrong about some things. Statistically, we're unlikely to be the outlier.

We have always been trying to sort pepples into "correct" heaps.[0]

[0] https://youtu.be/cLXQnnVWJGo

Post reply on HN