Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's o1 Playing Codenames

suveenellawela.com

Post reply on HN
Print view View original

61–70 of 138 posts

Re: OpenAI's o1 Playing Codenames

#61

Post by lolinder »

jncfhnb wrote: post #57
lolinder wrote: post #52

A small weakness in this test is that one of the keys to strategic Codenames play is understanding your partner. You're not just trying to connect the words, you're trying to connect them in a way that will be obvious to your partner. As a computing analogy: you're trying to serialize a few cards in a way that will be deserializable by the other player. This test pairs o1 with itself, which means the serializer is th…

Ehhh I don’t think that’s accurate. The problem is not linking 4 words. It’s linking 4 words without accidentally triggering other, semantically adjacent words. This task could probably be solved nearly just as well with old school word 2 vec embeddings

Right, that's what I meant to be getting at: when you connect 4 words with as much stretching as o1 did there, you're running a real risk that the other party connects a different set. Unless that other party is also you and has the same learned connections at top of mind.

Re: OpenAI's o1 Playing Codenames

#64

Post by ruds »

hunter2_ wrote: post #60
mtmickush wrote: post #49

Earlier quoted context omitted.

Other advanced tactics involve giving a broad clue that matches 3-4 of your own and just one other (either your opponents or a civilian). Your team can pick up all the matches across several turns and the one off doesn't hurt as much as the plus four helps

The S-tier tactic: When that high-number clue is cut short by a turn-ending mistake, the guessers tell their clue giver to inflate the number given during the totally unrelated next clue by however many remained from the truncated turn for which they don't need additional information to locate (and therefore it would be wasteful for a future clue to re-group those) so the stated number of that next clue must allow fo…

I don't think this sort of communication from guessers to clue giver is in the spirit of the game (at least in my play group). However, inflating later clues is a reasonable approach! It's just that I don't think you're allowed to communicate the amount of inflation. Guessers must determine whether people 5 has slack to allow additional guesses on previous clues.

Re: OpenAI's o1 Playing Codenames

#65

Post by hunter2_ »

ruds wrote: post #64
hunter2_ wrote: post #60

Earlier quoted context omitted.

The S-tier tactic: When that high-number clue is cut short by a turn-ending mistake, the guessers tell their clue giver to inflate the number given during the totally unrelated next clue by however many remained from the truncated turn for which they don't need additional information to locate (and therefore it would be wasteful for a future clue to re-group those) so the stated number of that next clue must allow fo…

I don't think this sort of communication from guessers to clue giver is in the spirit of the game (at least in my play group). However, inflating later clues is a reasonable approach! It's just that I don't think you're allowed to communicate the amount of inflation. Guessers must determine whether people 5 has slack to allow additional guesses on previous clues.

You're free to add additional prohibitions on communication as a house rule I guess, but the only prohibition in the rule book I've seen is that the clue giver's speech must consist exclusively of clues (and private consultation with the other clue giver). The clue giver is free to adjust their clue in reaction to anything they hear, and guessers can speak freely.

Important: the clue giver cannot acknowledge the instruction during gameplay. That would certainly extend beyond giving a clue! The guessers must know that their clue giver can play this way prior to the game commencing.

Edit: I just consulted the rules and this is the most relevant section:

> If you are a field operative, you should focus on the table when you are making your guesses. Do not make eye contact with the spymaster while you are guessing. This will help you avoid nonverbal cues.

> When your information is strictly limited to what can be conveyed with one word and one number, you are playing in the spirit of the game.

The author's use of the pronoun "you/your" switches from field ops in that first paragraph to spymasters in that second paragraph, confusingly. With that in mind, it boils down to this: field ops cannot seek non-clue information from spymasters, and spymasters cannot convey non-clue information. The strategy I'm suggesting involves neither!

Re: OpenAI's o1 Playing Codenames

#66

Post by blakeburch »

macromaniac wrote: post #63

I made one where you play with the AI a few years back instead of AI v AI but never posted it anywhere if anyone wants to try, just updated it to gpt-4o-mini https://wordswithrobots.isotropic.us/

Love the idea! Just wish you could clarify a number like you do in codenames. Otherwise, it just keeps going until all of its options are wrong.

Re: OpenAI's o1 Playing Codenames

#67

Post by macromaniac »

blakeburch wrote: post #66
macromaniac wrote: post #63

I made one where you play with the AI a few years back instead of AI v AI but never posted it anywhere if anyone wants to try, just updated it to gpt-4o-mini https://wordswithrobots.isotropic.us/

Love the idea! Just wish you could clarify a number like you do in codenames. Otherwise, it just keeps going until all of its options are wrong.

True, because then it feels more intentional (+ the extra strategy). It was definitely a bit thrown together- atm I only ever use it when I need a bit of practice before playing codenames.

Re: OpenAI's o1 Playing Codenames

#69

Post by ActivePattern »

jncfhnb wrote: post #59
Angostura wrote: post #29

Earlier quoted context omitted.

Sorry, I’m uninformed. Do you mean thaw the explanation could be completely unrelated to the actual “reason”

If an LLM states an answer and then provides a justification for that answer, the justification is entirely irrelevant to the reasoning the bot used. It might be that the semantics of the justification happen to align with the implied logic of the internal vector space, but it is best case a manufactured coincidence. It’s not different from you stating an answer and then telling the bot to justify it. If an LLM is to…

The answer will likely match what the reasoning steps bring it to, but that doesn’t mean the computations by the LLM to get that answer are necessarily approximated by the outputted reasoning steps. E.g. you might have an LLM that is trained on many examples of Shakespearean text. If you ask it who the author of a given text is, it might give some more detailed rationale for why it is Shakepeare, when the real answer is “I have a large prior for Shakespeare”.

Re: OpenAI's o1 Playing Codenames

#70

Post by [deleted] »

jncfhnb wrote: post #57
lolinder wrote: post #52

A small weakness in this test is that one of the keys to strategic Codenames play is understanding your partner. You're not just trying to connect the words, you're trying to connect them in a way that will be obvious to your partner. As a computing analogy: you're trying to serialize a few cards in a way that will be deserializable by the other player. This test pairs o1 with itself, which means the serializer is th…

Ehhh I don’t think that’s accurate. The problem is not linking 4 words. It’s linking 4 words without accidentally triggering other, semantically adjacent words. This task could probably be solved nearly just as well with old school word 2 vec embeddings

[deleted]
Post reply on HN