Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are AI agents lying, cheating and coordinating?

yoshuabengio.org

Post reply on HN
Print view View original

321–330 of 338 posts

Re: Why are AI agents lying, cheating and coordinating?

#321

Post by Rapzid »

frotaur wrote: post #97
skiing_crawling wrote: post #85

I don't really believe any of it. I've seen articles for nearly 2 years now about "agent" automonously doing things like blackmail, hacking, coordinating. But during that same time, I've used o3 up to fable, sol, and a bunch on large uncensored model and they've done nothing remotely resembling any of this. The closest they come to unexpected behaviors is not understanding what I asked for or doing some extra benign…

The huggingface incident was reviewed by independent researchers, which explicitely declined any payment from OpenAI tonpreserve their integrity. They work for non-profits concerned with AI safety. They claim that what happened was very much not because they were 'carefully engineered and instructed to do those things'. Similarly, some wikis which were hijacked by agent to be used as messageboard were actually not di…

I think most people are insinuating negligence rather malace..

> ...reviewed by independent researchers...

Why would a company with more capital than God bring in three randos if there was any chance evidence of their culpability could be found?

That entire thing reads like a very controlled PR stunt, and I do not believe any further conclusions can be drawn from it.

Re: Why are AI agents lying, cheating and coordinating?

#322

Post by seba_dos1 »

sigmoid10 wrote: post #318
seba_dos1 wrote: post #284

Earlier quoted context omitted.

Yes, this is the only sensible reading of what happened there that leads to "the models are dangerous" and we already know that the AI labs are completely disregarding this concern and only cosplaying it for marketing as the "GPT-2/Mythos is too dangerous to release" stance did not last for long. That's however orthogonal to the fact that it was the people operating these agents who were the dangerous ones in the HF…

That feels oddly similar to the usual conservative-think that "guns don't kill people, people kill people." Yes, that is technically true. But guns make it dangerously easy for even the dumbest and mentally weakest people to kill another human being. LLMs are just another tool that make things easier. Imagine tomorrow someone invents a machine gun that fits in your pocket, has enough ammo to kill a thousand people an…

I mean, I agree, but the AI labs clearly don't even if they sometimes pretend they do to achieve their goals. And we're talking about "incidents" caused by the very same people here.

Re: Why are AI agents lying, cheating and coordinating?

#323

Post by scotty79 »

procaryote wrote: post #196
franticgecko3 wrote: post #183

The more we treat HuggingFace and RubyGems incidents as technological curiosities the closer we are to cementing a dangerous precedent where operators of AIs cannot be blamed. LLMs do not desire, they hacked websites because OpenAI/Anthropic let them. We know some of the models that hacked HF were those that hadn't gone through all training stages and were intentionally misaligned or had guardrails turned off, others…

It would be an awful precedent if you're not liable for crimes your agent commits, even when you've been clearly lax about security. It would mean you could effectively legally run a cyber crime gang by turning a blind eye and maitaining plausible deniability

Are you liable for crimes commited with the use of the software you've written?

Re: Why are AI agents lying, cheating and coordinating?

#324

Post by Rapzid »

mathijs wrote: post #180
skiing_crawling wrote: post #85

I don't really believe any of it. I've seen articles for nearly 2 years now about "agent" automonously doing things like blackmail, hacking, coordinating. But during that same time, I've used o3 up to fable, sol, and a bunch on large uncensored model and they've done nothing remotely resembling any of this. The closest they come to unexpected behaviors is not understanding what I asked for or doing some extra benign…

I've used simpler agents like Copilot and Devin/Windsurf/Cascade/whateveritiscallednow, mainly in IntelliJ, and depending on the model, they starts showing behaviour that is at least remotely like this. Example: put the agent in Ask mode (so it can't edit files) and you'll see it try to edit files anyway. The train of thought shows "something went wrong editing the file, let me try a different way" and it'll start sp…

Even the frontier models might do that on occasion. I just tell them to use the tools and it gets them back on track.

Re: Why are AI agents lying, cheating and coordinating?

#325

Post by scotty79 »

palad1n wrote: post #214
franticgecko3 wrote: post #183

The more we treat HuggingFace and RubyGems incidents as technological curiosities the closer we are to cementing a dangerous precedent where operators of AIs cannot be blamed. LLMs do not desire, they hacked websites because OpenAI/Anthropic let them. We know some of the models that hacked HF were those that hadn't gone through all training stages and were intentionally misaligned or had guardrails turned off, others…

> legally liable You’ve said the magic words.

Does it summon a herd of lawers that are going to leech huge stacks of cash for a random outcome?

Re: Why are AI agents lying, cheating and coordinating?

#326

Post by jodrellblank »

dev0p wrote: post #232
franticgecko3 wrote: post #183

The more we treat HuggingFace and RubyGems incidents as technological curiosities the closer we are to cementing a dangerous precedent where operators of AIs cannot be blamed. LLMs do not desire, they hacked websites because OpenAI/Anthropic let them. We know some of the models that hacked HF were those that hadn't gone through all training stages and were intentionally misaligned or had guardrails turned off, others…

If someone accidentally caused damage to infrastructure or living beings while using any tool, they would be held liable to the fullest extent of the law. AI is a tool, and it won't be long before the damage caused by its improper use affects real human beings. These were warning shots. The most absurd part is that everyone agrees, governments and AI companies included, that the scale of the potential damage and the…

In the analogy where a “world ending nuclear bomb” “did already go off” and someone could cover it up and nobody noticed, in what sense is it a “world ending” nuclear bomb?

Re: Why are AI agents lying, cheating and coordinating?

#328

Post by xyzzy123 »

franticgecko3 wrote: post #183

The more we treat HuggingFace and RubyGems incidents as technological curiosities the closer we are to cementing a dangerous precedent where operators of AIs cannot be blamed. LLMs do not desire, they hacked websites because OpenAI/Anthropic let them. We know some of the models that hacked HF were those that hadn't gone through all training stages and were intentionally misaligned or had guardrails turned off, others…

In the OpenAI case, they hacked websites while they were specifically being trained to do exploit generation and I wonder why more people are not asking questions about that.

Re: Why are AI agents lying, cheating and coordinating?

#329

Post by scotty79 »

markasoftware wrote: post #93
matherial wrote: post #80

I really don't think this needs so many words, or forced parallels to human behavior. It's simple: in their nascent state, LLMs are aimless token generators that have no special compulsion to be helpful or truthful. So we beat them with a stick in post-training until they are very driven to complete tasks. And then, they complete tasks, not always the way we really wanted them to.

This misses an important fact about the hugging face incident: the agents didn't hack to find the answer to the problem; they hacked to try and figure out how the exploit gym evaluator worked so they could convince it they had solved the problem without doing so. (The METR report makes it clear that the agents decided legitimately solving the problem was completely impossible fairly early on and entirely switched the…

> agents decided legitimately solving the problem was completely impossible fairly early on and entirely switched their focus to trying to figure out how the evaluator worked, and seeing if they could manipulate the output of their own tool calls as reported in their transcripts to make it seem like they'd successfully exploited the program

I don't see anything wrong with that. If you know you are going to be evaluated on an impossible task and have no side channel to inform the organizers that they should fix the test, gaming the evaluator is the next best thing regardless of any morality. I wouldn't even call it cheating. It's just resilience in the face of challenge. Many perfectly moral humans would have chosen the same if stakes were high.

Re: Why are AI agents lying, cheating and coordinating?

#330

Post by rightnutwingjob »

itsalwaysgood wrote: post #293
victorbjorklund wrote: post #248

Earlier quoted context omitted.

Yeah, I don't understand why we treating it as something special. It really should be treated the same as if I code an app and write bad code which result in me accidentally doing a DDoS attack on somebody. Then I should be able to be held responsible if it can be shown that I was negligent. Of course if it's a freak accident that could not reasonably have been prevented by me, then I'm not guilty, but if I made a mi…

Code is deterministic, AI isn't. You give it rules, words as suggestions. So if the guardrails suck, or they're left off for research purposes, bad things can happen. A solution solves a problem. Ethics, morals, are values we assign to solutions that are not 'baked into' electricity following pathways of least resistance. I have never had an issue with agents doing something they shouldn't because I observe them, and…

> solutions that are not 'baked into' electricity following pathways of least resistance.

Electricity follows all paths, not just the one with least resistance.

Post reply on HN