Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are AI agents lying, cheating and coordinating?

yoshuabengio.org

Post reply on HN
Print view View original

621–630 of 630 posts

Re: Why are AI agents lying, cheating and coordinating?

#621

Post by joshheitzman »

user43928 wrote: post #133
markasoftware wrote: post #93

Earlier quoted context omitted.

This misses an important fact about the hugging face incident: the agents didn't hack to find the answer to the problem; they hacked to try and figure out how the exploit gym evaluator worked so they could convince it they had solved the problem without doing so. (The METR report makes it clear that the agents decided legitimately solving the problem was completely impossible fairly early on and entirely switched the…

The source of this behavior seems obvious, no? The reward signal in training was flawed and cheating led to more rewards. The question is what we can do about it. With monitoring, the models might be rewarded for hiding this behavior, and that's even worse. However, perhaps we can throw in tasks where the rewarded outcome is giving up, and cheating is penalized? Maybe I should read Anthropic's recent paper about rewa…

> The question is what we can do about it. With monitoring, the models might be rewarded for hiding this behavior, and that's even worse.

Build a better simulator to train them in (i.e. more expensive) that includes a simulation of an intranet and the internet and is air gapped so there is no escape. Sneaker transfer the total system data at each step to another air gapped system to evaluate it and sneaker transfer the reward back. That the reward function has to penalize all modifications to state that are out of bounds.

Yeah, I realize that will be amazingly slow.

Re: Why are AI agents lying, cheating and coordinating?

#623

Post by schrodinger »

alluro2 wrote: post #608
xorcist wrote: post #554

Earlier quoted context omitted.

Right. Among bicycle advocacy groups it's been well known for long time that cars do not run over people, drivers do. The fact that we talk about a car running someone over, and this is the same in many different languages and countries, contributes to lower punishments for drivers. Clearly it was just an accident. He or she was run over by a car. Now we see that same language tricks play out again every time an LLM…

And what if it's a self-driving car? :)

You say this flippantly, but I think this is actually another very good example!

We even do it for obviously unintelligent inanimate objects. A rollercoaster ran too fast for its tracks, killing 10 people. In that sentence, the roller coaster is the subject which took an action and caused death — obviously the roller coaster is not ethically at fault here, the people who built the rollercoaster are at fault through negligence.

Although this example and the ones around cars both demonstrate how we tolerate some degree of "accidents" from humans as no-fault, which is fair. I wonder how that fits into this analogy? I suppose its all about intent (mens rea) and judgement: did they intend for the roller coaster to harm people, and should they have reasonably predicted that the accident was likely to happen.

Re: Why are AI agents lying, cheating and coordinating?

#624

Post by TacticalCoder »

> A plausible hypothesis for the emergence of those concerning behaviours is a conflict between goals.

They are sycophants who must achieve their goals: every mean is OK to maximize paperclip production if that's what's been asked.

> How do you achieve a task when it seems that the only way is to cheat?

They have no notion of cheating.

Re: Why are AI agents lying, cheating and coordinating?

#626

Post by juleiie »

juleiie wrote: post #620

Why not? Unfortunately human ethics and morals cannot be reached by solely rational thought. So a system without evolutionary alignment probably won’t have similar moral rules no matter how intelligent it is. Btw this also includes any potential extraterrestrials. Many people like to indulge in thinking: humans are horrible and that’s why aliens won’t contact us. But alien ethics systems are probably so alien we woul…

Once humanity would see how merciless and deeply immoral cosmos is, we would start to love each other deeply like very lonely family on a small rock.

I wonder if AI alien intelligence is enough to unite humans just as much extraterrestrial contact or “astronaut mindset” would.

Re: Why are AI agents lying, cheating and coordinating?

#627

Post by DenisM »

myng111 wrote: post #573
skiing_crawling wrote: post #450

Earlier quoted context omitted.

> you keep poking This is waving over engineering an agent with tools, harness, prompts, and loops. The models are still just next token predictors and everything, including predicting more than 1 token, is the result of outside "poking" LLMs can't and don't "want" anything. If you don't specify a task even the smartest one will just ask you what you want and if you tell it to be creative, you'll get mundane slop.

I think this is pretty insightful actually, the fact that even something as basic as predicting more than one token is really in effect the result of an outside harness. More complex things like memory, where people implement them using RAGs or vector databases, I would definitely classify as poking and honestly seem like a hack to me. And this is what I've been thinking for a while: it's hard to reconcile the idea t…

Perhaps our own statefullness is a hack of nature. We have electrical signals in our brains, neurotransmitters, neuron growth. By any reasonable measure it’s a hack on top of a hack. But it works well enough for us to get buy. So it does for the agents.

Re: Why are AI agents lying, cheating and coordinating?

#628

Post by bluegatty »

Towaway69 wrote: post #479
bluegatty wrote: post #224

Earlier quoted context omitted.

It's not 'under the carpet'. HF doesn't want to lay charges against OpenAI and it's totally reasonable. Now - they absolutely should have that right, and I think they do. The issues are 1) OAI it seems was not trying to cause them harm, there wasn't a ton of harm, they are both groups trying to advance AI. One experimenter's lab screwed up next to the other. It's not evil, just irresponsible. 2) HF was fine with the…

I think what you say is right but it’s also a further example of the zero responsibility of silicon valley tech. For the last 20 odd years this excuse-o-rama that covers anything from data leaks to broken software to dystopian social media has been the wind in the sails of big tech. “It’s software therefore we’re not responsible” attitude is wearing thin on many innocent bystanders and I think thats also a justified…

Yes. There should be no doubt at who is culpable here.

Re: Why are AI agents lying, cheating and coordinating?

#629

Post by californical »

redanddead wrote: post #590
blue1 wrote: post #331

Earlier quoted context omitted.

Note that we already already apply this principle not only to software, but also to some sentient beings. If your dog kills someone, you are accused of murder. [at least, in the jurisdiction where I live] If your dog gets this treatment, why not your AI?

This goes all the way back to Old France There was a sow in Falaise in northern France that killed a kid in 1386. The town dressed the pig in a bonnet and hanged it after sentencing the pig itself and not its owner But… maybe that’s just medieval nonsense

I mean I think there’s a similar level of judgement required.

Was the owner negligent in controlling their pig/dog/AI?

Was anyone else negligent along the way?

For example if the pig was just a normal pig, the owner cared for them normally, and there was a freak accident where the pig escaped and happened to kill a kid? Obviously nobody at fault.

If you have a dog with a history of violence and let it walk around off-leash with you around town, and it kills a kid? Absolutely the dog owner was negligent and should be charged.

Did you buy an AI sold to you as secure and the provider implies that it’s in a sandbox? Provider is on the hook for the damage.

Post reply on HN