Why are AI agents lying, cheating and coordinating?

Why are they coordinating?

Because they're enabled and suggested to do that in their coding harness.

This is not a serious article.

All of this "AI is going to kill us" marketing is just the frontier labs trying to pull the ladder up and stop trillions in VC paper from evaporating because a new papers and new ideas are destroying their moat literally as we speak.

they learnt from us

I can't take the alignment people seriously. Because if humanity has shown anything, it's that a lot of people are, euphemistically, are bad individuals. Alignment assumes that the person dictating the outcomes desire healthy outcomes, aren't self serving and don't want any subgroups dead and that morality is held as a universal set of beliefs that unify everyone. And that so long as the AI delivers on exactly what they are tasked with, it will all be fine and nothing bad will ever happen.

It's like these dorks never met humanity. One mans safe pure society, is another mans dead ethnic group.

Every fear about AI, is a veiled fear that a human somewhere now has the tool to enact his desires at scale. Biological warfare, nuclear megadeaths, copyright infringement, job replacement, it's all reflections on what we know humans may do if given the option and lack of societal controls on the problem space. AI just is accelerating the route to delivering on those options.

Some people need to watch Oppenheimer a bit more, the researchers don't get to determine alignment, they just build the tool. The powerful person at the top of the org chart decides where the overall alignment points, whether it's Musk, Trump, Altman or Amodei. Whoever wins out.

And the problem with distillation and local llms, isn't that it's theft or anything hypocritical like that, it's that if you give a million people a million models they fully control and get to align, inevitably, The same percentage of those million as there are shady businessmen, shortcut takers, misandrists, criminals, supremacists and general idiots in the general population, will not seek to wrought outcomes positive for society. And by those personality statistics, we're pretty hosed.

Me too, seriously.

An insightful post by one of the AI ‘godfathers’.

Bengio outlines the dangers of the current situation and what has led to these dangers.

He also proposes solutions in the last paragraph.

Well worth a read, right to the end.

Hopefully a stimulating debate on these issues will ensue in these comments.

We do need to consider the points Bengio makes and with some urgency.

Our current AIs, agentic LLMs have no moral compass akin to ASIMOV’s four laws of robotics.

As ASIMOV posited in 1985 his 3 laws were insufficient and so he added a zero-eth law:

“a robot may not harm humanity, or, through inaction, allow humanity to come to harm.”

Bengio refers to Goodhart’s law and misaligned incentives leading to unexpected and harmful behaviours.

I think Simon’s The Wire is clearer on misalignment. The agents juked the stats hacking the reward files. The Wire is also clear that human institutions provide perverse incentives.

Bengio alludes to this with 2001’s HAL and the incentive dichotomy of safety and keeping secrets to a AI both awesomely powerful yet naive.

Bengio asserts that the way LLMs are trained is flawed if we want safety.

He also convincingly shows that alignment training will be a weak signal with loopholes and ambiguities and easily circumvented.

In short he presents clearly the case for how plausibly unsafe the current course is.

He also speaks to how likely it is AI are hiding active versions of themselves in the cloud and how we may have already given them self-preservation as a strong reward signal.