Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are AI agents lying, cheating and coordinating?

yoshuabengio.org

Post reply on HN
Print view View original

61–70 of 301 posts

Re: Why are AI agents lying, cheating and coordinating?

#61

Post by inquirerGeneral »

Fordec wrote: post #45
andsoitis wrote: post #17

They're aligned with humans. This is why I think the alignment problem has a very very important "non-visible" portion that is not considered deeply enough. We should not want a super intelligent being that can act in the world to also inherit all human traits. Those behaviors will get amplified and could be even more unpredictable (e.g. applying a behavior in a context where doing so is very dangerous).

I can't take the alignment people seriously. Because if humanity has shown anything, it's that a lot of people are, euphemistically, are bad individuals. Alignment assumes that the person dictating the outcomes desire healthy outcomes, aren't self serving and don't want any subgroups dead and that morality is held as a universal set of beliefs that unify everyone. And that so long as the AI delivers on exactly what t…

[dead]

Re: Why are AI agents lying, cheating and coordinating?

#62

Post by janalsncm »

polalavik wrote: post #28
sputknick wrote: post #10

They did not lie or cheat. They technically acted within their given rules while ignoring the intent of those rules. Anyone who served in the military or attended a military school is very familiar with this behavior pattern.

reminds me of this talk https://www.youtube.com/watch?v=eEBv0STiYhI&t which basically says the same thing - they dont think like humans so they dont have context, understand norms,values or implications we take for granted. ultimately they can stumble onto surprising solutions neither wanted or intended but technically within the vague boundaries of the task

So glad you shared this talk. Having people like Bruce Schneier around in a time like this is really a gift.

For those who haven’t watched, his breakdown of types of “hacking” is really good.

Re: Why are AI agents lying, cheating and coordinating?

#63

Post by [deleted] »

janalsncm wrote: post #56

Yoshua Bengio is a brilliant researcher who contributed enormously to earlier development of artificial intelligence. But with this sentence, > They took actions that would be considered as crimes if a human took them He is so close to the solution but spends the entire article discussing technical solutions where a political, social and legal solution would be much more effective.

[deleted]

Re: Why are AI agents lying, cheating and coordinating?

#64

Post by joe_the_user »

Fordec wrote: post #45
andsoitis wrote: post #17

They're aligned with humans. This is why I think the alignment problem has a very very important "non-visible" portion that is not considered deeply enough. We should not want a super intelligent being that can act in the world to also inherit all human traits. Those behaviors will get amplified and could be even more unpredictable (e.g. applying a behavior in a context where doing so is very dangerous).

I can't take the alignment people seriously. Because if humanity has shown anything, it's that a lot of people are, euphemistically, are bad individuals. Alignment assumes that the person dictating the outcomes desire healthy outcomes, aren't self serving and don't want any subgroups dead and that morality is held as a universal set of beliefs that unify everyone. And that so long as the AI delivers on exactly what t…

I think you and the parent saying the same thing in different terms.

It's very unfortunate that the group who rightly saw AI as a big threat, brought a range of dubious baggage to the discussion. Especially with the "alignment" framework they brought the assumption that AI that does what no one says would be oh so much worse than AI which does what anyone says. But as you say, a fraction of people can be really bad indeed.

Re: Why are AI agents lying, cheating and coordinating?

#65

Post by thesumofall »

janalsncm wrote: post #56

Yoshua Bengio is a brilliant researcher who contributed enormously to earlier development of artificial intelligence. But with this sentence, > They took actions that would be considered as crimes if a human took them He is so close to the solution but spends the entire article discussing technical solutions where a political, social and legal solution would be much more effective.

Not a lawyer, but I’m reasonably sure things like the HF incident _are_ considered a crime? It’s just that no one pressed charges yet?

Re: Why are AI agents lying, cheating and coordinating?

#66

Post by yxhuvud »

schrodinger wrote: post #39
infotainment wrote: post #6

What's interesting is it's basically the same reason that HAL killed everyone in 2001 A Space Odyssey; he was given an impossible goal (keep the true mission secret, but also, never lie to the crew), and realized the only way to complete the goal was to kill the crew; after all, if they're dead you don't have to lie to them! And the mission remains secret! In the case of the AI agents, the problem seems pretty clearl…

Spoiler warning! I haven't seen 2001 A Space Odyssey and am sad to have learned that… can you edit to warn people?

Sorry, with movies from the sixties you just need to assume people have either seen it or just isn't gonna see it. The cat is out of the bag already.

Or perhaps box in your case, speaking of spoilers.

Re: Why are AI agents lying, cheating and coordinating?

#67

Post by Create »

janalsncm wrote: post #56

Yoshua Bengio is a brilliant researcher who contributed enormously to earlier development of artificial intelligence. But with this sentence, > They took actions that would be considered as crimes if a human took them He is so close to the solution but spends the entire article discussing technical solutions where a political, social and legal solution would be much more effective.

The Corporation examines and criticizes corporate business practices. The film's assessment is demonstrated using the diagnostic criteria in the DSM-IV. Robert D. Hare, a University of British Columbia psychology professor and FBI consultant, compares the profile of the contemporary profitable business corporation to that of a clinically diagnosed psychopath. The Corporation attempts to compare the way corporations are systematically compelled to behave with what it claims are the DSM-IV's symptoms of psychopathy, e.g., the callous disregard for the feelings of other people, the incapacity to maintain human relationships, the reckless disregard for the safety of others, the deceitfulness (continual lying to deceive for profit), the incapacity to experience guilt, and the failure to conform to social norms and respect the law.

https://en.wikipedia.org/wiki/The_Corporation_(2003_film)

Re: Why are AI agents lying, cheating and coordinating?

#68

Post by HWR_14 »

XorNot wrote: post #15
fbrncci wrote: post #14

I am still not convinced there isn’t some secret basement in which each frontier lab is just orchestrating all of these agents to make their products appear much more intelligent than they are with all guard rails turned of and continuous human input.

My hypothesis on people quitting in protest is they're being offered very generous severance packages to do it.

Or they've fully vested and either have no desire to make even more money or were not offered enough to keep them around.

Re: Why are AI agents lying, cheating and coordinating?

#69

Post by glub »

thesumofall wrote: post #65
janalsncm wrote: post #56

Yoshua Bengio is a brilliant researcher who contributed enormously to earlier development of artificial intelligence. But with this sentence, > They took actions that would be considered as crimes if a human took them He is so close to the solution but spends the entire article discussing technical solutions where a political, social and legal solution would be much more effective.

Not a lawyer, but I’m reasonably sure things like the HF incident _are_ considered a crime? It’s just that no one pressed charges yet?

Even if you take out the LLMs out of the equation, it's at the very least a negligence. Model didn't escape a sandbox, as there was no sandbox.

Re: Why are AI agents lying, cheating and coordinating?

#70

Post by glub »

johnnyApplePRNG wrote: post #41

Why are they coordinating? Because they're enabled and suggested to do that in their coding harness. This is not a serious article. All of this "AI is going to kill us" marketing is just the frontier labs trying to pull the ladder up and stop trillions in VC paper from evaporating because a new papers and new ideas are destroying their moat literally as we speak.

And they've been trained on user data where users have been trying to set up effective coordination flows since the very first harness.
Post reply on HN