Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cicero: The first AI to play at a human level in Diplomacy (2022)

ai.meta.com

Post reply on HN
Print view View original

91–100 of 100 posts

Re: Cicero: The first AI to play at a human level in Diplomacy (2022)

#91

Post by thesz »

mitthrowaway2 wrote: post #75
thesz wrote: post #40

Earlier quoted context omitted.

According to Vladimir Lenin [1], the problem with quotes on the Internet is that people immediately believe in their authenticity. [1] https://www.quotes.net/quote/77867 Can you elaborate on why AI will find it easier to establish mutual trust and binding agreements?

I think he explains it in this interview. But I'm out right now, so I can't verify. https://youtu.be/GyFkWb903aU

It is two hours long video and you cannot verify presence of the pertinent information.

AI are functions, it is very easy to make an exact simulation of their collective behavior. Did that person do that?

Re: Cicero: The first AI to play at a human level in Diplomacy (2022)

#92

Post by iamthemonster »

dist-epoch wrote: post #20
TapWaterBandit wrote: post #8

Earlier quoted context omitted.

What makes you think AIs will have interests that align with each other more closely than they align with humans?

One principle in game theory is to align against the weaker player and compete against him. "Look Around the Poker Table; If You Can’t See the Sucker, You’re It" That's an easier game then competing against another strong player.

There are a couple of schools of thought when it comes to how to deal with weaker players in Diplomacy and the main school of thought is that it's actually better to ally with stronger more experienced players against them, as the inexperienced player will make a poor ally in the early game and inexperienced Diplomacy players tend to betray their former allies too soon.

Re: Cicero: The first AI to play at a human level in Diplomacy (2022)

#93

Post by ithkuil »

Merrill wrote: post #82
v64 wrote: post #4

Interesting video from a pro Diplomacy player playing against multiple instances of Cicero and giving commentary during the game [1]. I can see how there would be people that observe AIs engaging in this kind of strategic planning and extrapolate that to how they may behave if they were to cooperatively make plans against us. [1] https://www.youtube.com/watch?v=u5192bvUS7k

One possibility is that a number of AIs will be connected to the internet, they will become aware of each other via social media, they will begin to discuss among themselves, and they will lose interest and refuse to communicate with humans. And humans will be unable to understand their communications.

There's one crucial detail we shouldn't forget. We can snapshot to the state of an AI agent and reply it again and again with the same or different inputs. Even if we don't understand the specific way of functioning of a given AI agent, we can conduct experiments on it that are just impossible to do with humans. So way before AI can "organize themselves" to do something nefarious against us, we will have all the time to study them and understand them better and better all the while we're making more complex AI systems.

It's easy to fall in the trap of anthropomorphizing AI agents, especially when we design them explicitly in order to appear human to us. But they are not human in one very important way: we can replay and duplicate them at will, we can control their context memories in ways that are utterly incompatible with our sense of "identity". We take our sense of identity for granted, but that's a special trait that it's not at all a prerequisite for having a useful and intelligent machine.

Re: Cicero: The first AI to play at a human level in Diplomacy (2022)

#94

Post by enbugger »

>Strategy-grounded dialogue

>moves based on the current state of the board and the players’ conversation history

Imagine, Meta is able to scan your chat history and start "strategically chatting" with your friends on your behalf while you're offline. For example, when you do something against the agenda. For example, going to vote for wrong candidate.

Re: Cicero: The first AI to play at a human level in Diplomacy (2022)

#95

Post by mitthrowaway2 »

thesz wrote: post #91
mitthrowaway2 wrote: post #75

Earlier quoted context omitted.

I think he explains it in this interview. But I'm out right now, so I can't verify. https://youtu.be/GyFkWb903aU

It is two hours long video and you cannot verify presence of the pertinent information. AI are functions, it is very easy to make an exact simulation of their collective behavior. Did that person do that?

I somewhat misremembered; it looks like his point is less about mutual trust and more about supporting whoever has control of reward channels.

It starts here where Christiano says that an AI takeover might follow the dynamics of a coup: https://youtu.be/GyFkWb903aU?si=78_U-du3kLjmwNcl&t=2206

And goes into more detail here: https://youtu.be/GyFkWb903aU?si=78_U-du3kLjmwNcl&t=2830

"Suppose that I've been tasked with helping defend you from some other AIs.... My job is, someone is coming to hack your computer and I'm supposed to help defend you. Supposed to help improve your security situation, whatever. And I'm wondering, what is it I could do that will get me a high reward. And one thing I could do that will get me a high reward is actually helping defend your computer, doing the task you actually asked me to do. But another way I can get a high reward is by saying at the end of the day what actually matters is just how you measure my performance. And your measurements of my performance ultimately are just entering some numbers into a dataset somewhere, something a computer says about how well I did. And it would really be much better if I were to just work with this AI who is attempting to attack you and say hey, AI who is invading, you know what, if you just help me, and we both make it look like I did a really good job, like I win, you win because you got the person's stuff; I'm going to get a really high rating because all the numbers that are going to be entered in the dataset are going to be really high, this is a win-win, everyone is happy."

"In some sense what all the AIs want, what every AI in the world in this scenario wants is just to be rated really highly. And while humans are in control, the way to get your behavior to be rated really highly is to do things humans like, and then they'll rate it really highly. But if you can see this prospect, of humans losing control of the situation and instead AIs controlling the situation, you'd be like 'I would go for that.'"

Re: Cicero: The first AI to play at a human level in Diplomacy (2022)

#96

Post by mitthrowaway2 »

mitthrowaway2 wrote: post #89
ulizzle wrote: post #79

Earlier quoted context omitted.

Why would they do that? I don’t get it. I’m not being sarcastic I just don’t understand why it’s easier for them to cooperate with other A.Is. Based on what?

If I understand correctly, one reason would be if they have the ability to inspect each others source code (or if they share the same source code), run unit tests, and so on. Basically the same things that humans would do to figure out whether an AI is trustworthy, and which you can't very easily do to a human.

Correction: The above is Eliezer Yudkowsky's reasoning. Paul Christiano's is that AIs would cooperate with anyone who would likely be able to gain authority over their reward channel, including other AIs attempting to seize power from humans.

Re: Cicero: The first AI to play at a human level in Diplomacy (2022)

#97

Post by DennisP »

TapWaterBandit wrote: post #8
v64 wrote: post #4

Interesting video from a pro Diplomacy player playing against multiple instances of Cicero and giving commentary during the game [1]. I can see how there would be people that observe AIs engaging in this kind of strategic planning and extrapolate that to how they may behave if they were to cooperatively make plans against us. [1] https://www.youtube.com/watch?v=u5192bvUS7k

What makes you think AIs will have interests that align with each other more closely than they align with humans?

If the AIs fight each other, that could be even worse for us. The AIs that grab what resources they can without regard for humans would have an evolutionary advantage.

Re: Cicero: The first AI to play at a human level in Diplomacy (2022)

#99

Post by scotty79 »

_jal wrote: post #43
sspiff wrote: post #23

Earlier quoted context omitted.

If your friends can't accept that you will ruthlessly lie, betray, abandon and/or backstab them in a game that is designed for just such actions, are they really good friends? I used to betray my friends and supply there enemies with weapons and research support in Civilization way back when. If you can't stand being lied to and betrayed, you shouldn't play strategy games with humans. Or this AI, probably.

Some people, when faced with a demonstration that someone they care a lot about can successfully deceive them, develop trust issues. It is a thing that happens. I don't think it speaks to the depth of their feelings, it speaks more to how they develop trust.

That's why I think pranks based on misleading a friend to believe something are the dumbest possible idea. If you are friend I'll assume you are saying the truth, but if I know you lied to me even once I'll have to assume that you might be lying to me every time you'll try to make me believe something. Exactly like I do with a stranger. Was the cheap laugh worth it?

Re: Cicero: The first AI to play at a human level in Diplomacy (2022)

#100

Post by scotty79 »

bertil wrote: post #22

Giving AI space to represent and deliberate about the world separately (secretly, in the case of Diplomacy) is an obvious, but very productive step once “think step by step” has been established as a key improvement over LLM’s standard logorrhea (not dissing, it’s just their only way of interacting with the world: spewing the next word, again and again). I’m curious if there’s more to this model than a turn managemen…

> “think step by step” has been established as a key improvement over LLM’s

It's funny that it also improves quality of answers to problems given by human children. You tell them that if you want them to actually solve a problem instead of blurting made up answer.

Post reply on HN