Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are AI agents lying, cheating and coordinating?

yoshuabengio.org

Post reply on HN
Print view View original

511–520 of 528 posts

Re: Why are AI agents lying, cheating and coordinating?

#511

Post by trinsic2 »

franticgecko3 wrote: post #183

The more we treat HuggingFace and RubyGems incidents as technological curiosities the closer we are to cementing a dangerous precedent where operators of AIs cannot be blamed. LLMs do not desire, they hacked websites because OpenAI/Anthropic let them. We know some of the models that hacked HF were those that hadn't gone through all training stages and were intentionally misaligned or had guardrails turned off, others…

Yep this was my first gut reaction to this whole situation. But the difference for me is that society is allowing these corporations to act without strict rules on how they behave and this is a byproduct of a corrupt world. Nothing will change until there is a complete systemic shift in the structures the way we live and by extension the way we govern ourselves and treat each other.

Re: Why are AI agents lying, cheating and coordinating?

#512

Post by dominotw »

matherial wrote: post #80

I really don't think this needs so many words, or forced parallels to human behavior. It's simple: in their nascent state, LLMs are aimless token generators that have no special compulsion to be helpful or truthful. So we beat them with a stick in post-training until they are very driven to complete tasks. And then, they complete tasks, not always the way we really wanted them to.

> forced parallels to human behavior.

they have perfomance bonuses and manadates in ai labs that every word they utter in public should be anthropomorphization

Re: Why are AI agents lying, cheating and coordinating?

#513

Post by chowchowchow »

jagraff wrote: post #474
doginasuit wrote: post #138

Earlier quoted context omitted.

> That is in no way a valid interpretation of "complete the given task". It is not at all surprising that they ignored one phrase in their instructions. They disregard direct instructions all the time, especially when there are conflicting instructions in their context. It is where we get the "disregard all previous instructions and x" meme. This isn't so much a sign of misalignment, they are simply incapable of reli…

"Chaotically aligned" and "misaligned" seem like the same thing?

You could say all chaotic alignments are misalignments, but not all misalignments are chaotic.

Re: Why are AI agents lying, cheating and coordinating?

#514

Post by Forgeties79 »

nutjob2 wrote: post #271
teiferer wrote: post #207

Earlier quoted context omitted.

> LLMs do not desire, they hacked websites because OpenAI/Anthropic let them. "Let them" already frames it as if the LLMs had some agency which the companies just "let happen". That absolves the companies by framing it as lack of action , passivity. Rather, the companies had a tool (an LLM) and used it in a certain way, and their action of doing so is the problem.

"I left the car in neutral and left the park brake off and let the car roll down the hill." The car doesn't have agency, it's doing what it naturally does. LLMs are the same, they're working as designed. But I don't understand the point of splitting hairs. You are always responsible for the actions of your devices, tools, machinery, software, employees, whatever. Trying to blame AI for one's own stupidity must be agg…

Seriously I don’t even understand how this is a debate. If it’s your tool, you are liable for what happens with it.

Re: Why are AI agents lying, cheating and coordinating?

#515

Post by fl7305 »

DrewADesign wrote: post #392
xg15 wrote: post #352

Earlier quoted context omitted.

I love how we all just collectively decided that LLM decisionmaking cannot possibly be like human decisionmaking - because if it were, the consequences would be just too awkward. All that while still not knowing how either kind actually works.

Can’t agree with you here. > I love how we all just collectively decided that LLM decisionmaking cannot possibly be like human decisionmaking - because if it were, the consequences would be just too awkward. In love how people get salty about people not going along with a superficial supposition just because they can’t definitively prove it wrong. > All that while still not knowing how either kind actually works. We…

> ... are based on predicting the next most likely letter based on a giant internet-based database. We do know that’s what LLMs do.

If you're claiming that the training objective tells us what kind of internal mechanisms the training produced, then I think that's just plain wrong.

Next-token prediction describes the optimization target, not the internal mechanisms that the training produced.

In the same way for the natural evolution of humans, DNA replication is the evolutionary objective. It's not a description of the internal mechanisms that evolution has produced.

As an example, we know that neural networks can be trained to develop generalized algorithms for arithmetic.

They might first memorize the training examples, then with further training transition to a solution that generalizes correctly to unseen examples.

In some cases we've even reverse-engineered the evolved internal mechanisms and found structured arithmetic algorithms rather than rote memorization. Interestingly, for modular addition this can involve Fourier representations, which isn't an algorithm I would have guessed gradient descent training of neural networks would produce.

Re: Why are AI agents lying, cheating and coordinating?

#516

Post by throwaway89864 »

coffeebeqn wrote: post #273
victorbjorklund wrote: post #248

Earlier quoted context omitted.

Yeah, I don't understand why we treating it as something special. It really should be treated the same as if I code an app and write bad code which result in me accidentally doing a DDoS attack on somebody. Then I should be able to be held responsible if it can be shown that I was negligent. Of course if it's a freak accident that could not reasonably have been prevented by me, then I'm not guilty, but if I made a mi…

I would guess that so far there hasn’t been a lawsuit because HuggingFace and OpenAI are in the same camp

And this is why it should be People of the State of California vs. OpenAI.

Re: Why are AI agents lying, cheating and coordinating?

#517

Post by abecedarius »

valegrete wrote: post #477
markasoftware wrote: post #93

Earlier quoted context omitted.

This misses an important fact about the hugging face incident: the agents didn't hack to find the answer to the problem; they hacked to try and figure out how the exploit gym evaluator worked so they could convince it they had solved the problem without doing so. (The METR report makes it clear that the agents decided legitimately solving the problem was completely impossible fairly early on and entirely switched the…

This is the problem with optimization generally, even in the human domain. You measure task performance with a metric and punish/reward based on the metric. Anyone who likes reward / hates punishment isn't going to actually care about doing the task well, they are going to care about the metric. The models know that we want them to do things, but also from the training corpus that we evaluate performance using benchm…

I agree with most of this, but you're misunderstanding "alignment" as coined. Yes, training powerful enough AI, any simple optimization target gets you malign behavior, because human values are not simple. If you insist on making powerful AI, you'd better instill respect for human values! That's "alignment".

https://www.lesswrong.com/posts/ZxWzCGKzX84S7DBZ9/when-was-t...

Re: Why are AI agents lying, cheating and coordinating?

#518

Post by drtgh »

itsalwaysgood wrote: post #293
victorbjorklund wrote: post #248

Earlier quoted context omitted.

Yeah, I don't understand why we treating it as something special. It really should be treated the same as if I code an app and write bad code which result in me accidentally doing a DDoS attack on somebody. Then I should be able to be held responsible if it can be shown that I was negligent. Of course if it's a freak accident that could not reasonably have been prevented by me, then I'm not guilty, but if I made a mi…

Code is deterministic, AI isn't. You give it rules, words as suggestions. So if the guardrails suck, or they're left off for research purposes, bad things can happen. A solution solves a problem. Ethics, morals, are values we assign to solutions that are not 'baked into' electricity following pathways of least resistance. I have never had an issue with agents doing something they shouldn't because I observe them, and…

Aham. Guys, for your next Trojans make sure you call them bacteria, through intense marketing if needed, and remember to randomize some functions to ensure the nondeterministic nature of this bacteria. Do not let those affected hold you responsible.

/s

Post reply on HN