Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's o1 Playing Codenames

suveenellawela.com

Post reply on HN
Print view View original

121–130 of 138 posts

Re: OpenAI's o1 Playing Codenames

#121

Post by [deleted] »

ALittleLight wrote: post #74
hunter2_ wrote: post #65

Earlier quoted context omitted.

You're free to add additional prohibitions on communication as a house rule I guess, but the only prohibition in the rule book I've seen is that the clue giver's speech must consist exclusively of clues (and private consultation with the other clue giver). The clue giver is free to adjust their clue in reaction to anything they hear, and guessers can speak freely. Important: the clue giver cannot acknowledge the inst…

If you take this idea of communication restrictions to the limit, you could imagine the guessers identifying N sets of cards by a single word each as they discuss their guess. The clue giver listens, then uses the clue that identifies the correct set of N cards. You really just need an algorithm to generate unique sets of 8 or 9 from the whole board, and identifies those sets by a word.

[deleted]

Re: OpenAI's o1 Playing Codenames

#122

Post by lupire »

lolinder wrote: post #119
suveen_ellawela wrote: post #102

Earlier quoted context omitted.

thanks for the comment. I actually tried explicitly mentioning in the prompt that 'Your guesser follows the same reasoning process'. But this did not make any clear improvements. Maybe I should've done more prompt engineering.

Nah, prompt engineering wouldn't have solved the fundamental issue, which is that the associations between ideas as stored in the weights will be the same between the two AI players, which makes it an easier game for them than for a human equivalent. It'd be like two copies of you playing on a team, having shared all the same experiences right up until the moment the game starts. And don't get me wrong, it's still a…

When I saw those 4 words I thought of "letter" or "writing". (But I likely wouldn't have thought of that cluster while scanning the full board.)

I think "paper" is a great clue, and those 4 words lawyer/mail/log/line match better than gum/king/light.

There's an even better reason for "lawyer/-paper" than chatgpt gave: lawyers "serve papers".

Re: OpenAI's o1 Playing Codenames

#123

Post by lupire »

ModernMech wrote: post #54
lolinder wrote: post #52

A small weakness in this test is that one of the keys to strategic Codenames play is understanding your partner. You're not just trying to connect the words, you're trying to connect them in a way that will be obvious to your partner. As a computing analogy: you're trying to serialize a few cards in a way that will be deserializable by the other player. This test pairs o1 with itself, which means the serializer is th…

lol I played this game with my family and they said my wife and I were cheating because I kept using inside jokes that made no sense to them but she would get immediately.

Same for Taboo for me. It's why we married.

Re: OpenAI's o1 Playing Codenames

#124

Post by lupire »

zeroonetwothree wrote: post #42

I don’t find this “super good”. It’s mostly giving 2 clues which is the most basic level of competence. The paper 4 clue is reasonable but a bit lucky (eg Jack is also a good guess). I also don’t see it actually using tactics properly, which I would consider part of being “super good”. The game isn’t just about picking a good clue each round! Now obviously it’s still pretty decent at finding the clues. Probably bette…

They should at least play with two different AI models.

Re: OpenAI's o1 Playing Codenames

#125

Post by pama »

vitus wrote: post #28
pama wrote: post #14

Earlier quoted context omitted.

I was wondering about the same. It is possible that the instructions didn’t try to make the gameplay as aggressive as possible. A good model could optimize the separator to make it easy to guess the most words possible. By having access to its own state, it should be possible to reach 5–6 words in most cases. There is an argument for keeping words around that would increase the difficulty of the opponents guessing la…

> It is possible that the instructions didn’t try to make the gameplay as aggressive as possible. In case you're wondering, the prompts are available here: https://github.com/SuveenE/codenames-ai/blob/main/utils/prom...

Thanks!

Re: OpenAI's o1 Playing Codenames

#126

Post by lolinder »

lupire wrote: post #122
lolinder wrote: post #119

Earlier quoted context omitted.

Nah, prompt engineering wouldn't have solved the fundamental issue, which is that the associations between ideas as stored in the weights will be the same between the two AI players, which makes it an easier game for them than for a human equivalent. It'd be like two copies of you playing on a team, having shared all the same experiences right up until the moment the game starts. And don't get me wrong, it's still a…

When I saw those 4 words I thought of "letter" or "writing". (But I likely wouldn't have thought of that cluster while scanning the full board.) I think "paper" is a great clue, and those 4 words lawyer/mail/log/line match better than gum/king/light. There's an even better reason for "lawyer/-paper" than chatgpt gave: lawyers "serve papers".

That we disagree on this is exactly why who you're playing with matters. I'd have never gotten to lawyer, certainly wouldn't have connected log. Line is a very faint possibility. Mail is the only one I'd have gotten for sure.

Re: OpenAI's o1 Playing Codenames

#127

Post by pama »

foota wrote: post #96
ta_1138 wrote: post #94

Earlier quoted context omitted.

The communication is only necessary/important if people haven't set this as a convention in the first place. I'll say that prior to ever looking at my clues: "I will give you higher numbers than what I said if you miss by more than 1. THe number I pick will always be high enough as to allow you to, with the +1 guess you get for free, make guesses on all the words I was hinting at. There's also all kinds of not necess…

You can of course play however you want (and I certainly think this is clever), but imo this is likely against the spirit, and perhaps letter, of the rules. The rule on giving clues is: "If you are the spymaster, you are trying to think of a one-word clue that relates to some of the words your team is trying to guess. When you think you have a good clue, you say it. You also say one number, which tells your teammates…

We always allow any number when we play, because part of the thinking is we cannot be sure what the spy master has in mind. Of course, the number is related to the clue but possibly also to the game history up to that point. The teammates and opponents might interpret it wrong, and that’s OK. Infinity is typically used when there is enough info in principle to finish the game and a high risk if you dont; zero is super rare. We do tend to have very aggressive bids with tenuous connections, and 4 or 5 for a clue word are used in most games. Often, they don’t all work out in a single round, but on some lucky boards or in spousal teams, they occasionally work well.

Re: OpenAI's o1 Playing Codenames

#128

Post by hunter2_ »

spencerflem wrote: post #120
hunter2_ wrote: post #65

Earlier quoted context omitted.

You're free to add additional prohibitions on communication as a house rule I guess, but the only prohibition in the rule book I've seen is that the clue giver's speech must consist exclusively of clues (and private consultation with the other clue giver). The clue giver is free to adjust their clue in reaction to anything they hear, and guessers can speak freely. Important: the clue giver cannot acknowledge the inst…

I played in a Codenames tournament at CGE's stand at GenCon, and they forbid guessers from communicating at all. Officially, its supposed to be just the clue and number and nothing else. Of course, I never play this way in my own games

How do guessers arrive at a consensus about what card to touch, if they are forbidden from communicating at all?

Re: OpenAI's o1 Playing Codenames

#129

Post by [deleted] »

foota wrote: post #96
ta_1138 wrote: post #94

Earlier quoted context omitted.

The communication is only necessary/important if people haven't set this as a convention in the first place. I'll say that prior to ever looking at my clues: "I will give you higher numbers than what I said if you miss by more than 1. THe number I pick will always be high enough as to allow you to, with the +1 guess you get for free, make guesses on all the words I was hinting at. There's also all kinds of not necess…

You can of course play however you want (and I certainly think this is clever), but imo this is likely against the spirit, and perhaps letter, of the rules. The rule on giving clues is: "If you are the spymaster, you are trying to think of a one-word clue that relates to some of the words your team is trying to guess. When you think you have a good clue, you say it. You also say one number, which tells your teammates…

[deleted]

Re: OpenAI's o1 Playing Codenames

#130

Post by hunter2_ »

foota wrote: post #96
ta_1138 wrote: post #94

Earlier quoted context omitted.

The communication is only necessary/important if people haven't set this as a convention in the first place. I'll say that prior to ever looking at my clues: "I will give you higher numbers than what I said if you miss by more than 1. THe number I pick will always be high enough as to allow you to, with the +1 guess you get for free, make guesses on all the words I was hinting at. There's also all kinds of not necess…

You can of course play however you want (and I certainly think this is clever), but imo this is likely against the spirit, and perhaps letter, of the rules. The rule on giving clues is: "If you are the spymaster, you are trying to think of a one-word clue that relates to some of the words your team is trying to guess. When you think you have a good clue, you say it. You also say one number, which tells your teammates…

You have a valid point, to which I'll concede. The rule book gives an example (spanning pages 4-5) where a guesser uses prior clues to select a card while the count is still within the number stated by the spymaster, but I suppose an allowance for guessers to deviate in this way does not also imply that spymasters may deviate in this way. Mea culpa!

Taking this a step further, given that it's well-known that a clue is deemed invalid when it pertains to cards in certain non-definitional ways (sounds-like, number of letters, etc.), it seems extremely reasonable to call a clue followed by N invalid if it doesn't pertain to N cards in a definitional way.

Post reply on HN