Live data from Hacker News

CICERO: An AI agent that negotiates, persuades, and cooperates with people

ai.facebook.com

231–240 of 300 posts

Re: CICERO: An AI agent that negotiates, persuades, and cooperates with people

#231
post #226

Earlier quoted context omitted.

Comments like this make me excited about the book I'm currently writing (hopefully the nail on the head at the tip of the zeitgeist), egregores and tulpas are primary concepts/characters.

Would you consider money as a form of egregores? Money seems to have a live of its own full of money-live support systems, ie. buyers and sellers.

I think Bitcoin is pretty great example of an egregores

Re: CICERO: An AI agent that negotiates, persuades, and cooperates with people

#232
post #199
post #186

Earlier quoted context omitted.

"All is fair in Love and Civ" is a common variant of the default rules. It introduces the "your box has been pwned" victory/loss condition.

Is this “all’s fair” stance an actual accepted thing? If so, what are the limits on it, can I assassinate someone? Start an actual war to cause a player to resign? Blow up their planet? Clearly there’s a tacit assumption of some boundary around the game.

[deleted]

Re: CICERO: An AI agent that negotiates, persuades, and cooperates with people

#234
post #4

"In a game in which dishonesty is commonplace, it is notable that we were able to achieve human-level performance by controlling the agent's dialogue through the strategic reasoning module to be largely honest and helpful to its speaking partners."

Well, yes, "largely" honest, until the critical moment when you stab to take 18 supply centers to win the game. You don't win Diplomacy by staying honest, you win by choosing the right moment to make the move and burn that trust. It's the 5% dishonesty that matters.

So Diplomacy is just an acrimonious version of iterated prisoner's dilemma?

Re: CICERO: An AI agent that negotiates, persuades, and cooperates with people

#235
post #20

Paper: https://www.science.org/doi/10.1126/science.ade9097 Code: https://github.com/facebookresearch/diplomacy_cicero Site: https://ai.facebook.com/research/cicero/ Expert player vs. Cicero AI: https://www.youtube.com/watch?v=u5192bvUS7k RFP: https://ai.facebook.com/research/request-for-proposal/toward... The most interesting anecdote I heard from the team: "during the tournament dozens of human players never even su…

"Having read the paper & supplementary materials, watched narrated game & spoken to one of the human players I'm pretty concerned. The @ScienceMagazine paper centres 'human-AI cooperation' & the bot is not supposed to lie. However, videos clearly show deception/manipulation"

"Screenshots of the stab below.

The human player said: "The bot is supposed to never lie [...] I doubt this was the case here" "I was definitely caught more off guard as a result of this message; I knew the bot doesn't lie, so I thought the stab wouldn't happen." "

"I'd like the researchers involved to say quite a bit more about "A.3 Manipulation"

What are possible prevention, detection & mitigation steps?

What are the possible use cases? What are the benefits/downsides of them? Has Meta considered developing products based on this?" -- Haydn Belfield, a Cambridge University researcher who focuses on the security implications of artificial intelligence (AI).

https://twitter.com/HaydnBelfield/status/1595168102924402688

https://www.cser.ac.uk/team/haydn-belfield/

Re: CICERO: An AI agent that negotiates, persuades, and cooperates with people

#236

Earlier quoted context omitted.

I suspect we disagree on whether (&/or what) should be done about it, if anything.

Disagreeing would require me to have a level of fixed perspective and permanent identity that I think you're assuming that I have.

Eh? How come? People disagree all the time, and I suspect not all that many people have a level of fixed perspective and permanent identity.

Re: CICERO: An AI agent that negotiates, persuades, and cooperates with people

#237
post #209

One of the guys who created this also created Pluribus, the AI that learned to bluff to win poker games. That's Noam Brown. https://www.cmu.edu/ambassadors/october-2019/artificial-inte... High-level, you need three things for AI to get "ex machina"-level creepy: 1) the ability to successfully manipulate humans to attain its ends 2) the ability to rewrite its objective function; ie to redefine its ends. 3) a multi-mod…

I'm not sure 2 is necessary. As long as the objective function is sufficiently far away the AI has a lot of breadth to achieve it.

But based on the paper, it sounds like this model is lacking in 3. I'm curious as well how far we are from a more general model that is able to achieve the same results. From the development of generative AI, we might not be that far.

Re: CICERO: An AI agent that negotiates, persuades, and cooperates with people

#238
post #194
post #106

Earlier quoted context omitted.

A "binding" treaty where the terms of the treaty are supposedly binding, but, like on the real world stage, the only penalties for breaking them is a lowered trust from all the other countries/players, sounds fine. A binding treaty where if you break them someone will say "that's against the rules of the game, and the rest of us will stop playing if you don't comply" sounds terrible.

> someone will say "that's against the rules of the game, and the rest of us will stop playing if you don't comply" sounds terrible. And yet, that’s how the USD is going to lose its reserve currency status in the next few decades, if not considerably sooner.

No post body was provided.

Re: CICERO: An AI agent that negotiates, persuades, and cooperates with people

#240
Great research direction you guys!

If I may propose some other project ideas for you:

- What if we could teach gorillas to wield machetes?

- What if we could figure out how to craft a nuclear weapon out of common office supplies?

- What if we could make AIDS airborne and contageous?

Post reply on HN