Live data from Hacker News

Which AI Lies Best? A game theory classic designed by John Nash

so-long-sucker.vercel.app

71–80 of 83 posts

Re: Which AI Lies Best? A game theory classic designed by John Nash

#71

Earlier quoted context omitted.

First try with claude: https://claude.ai/share/fabaf585-3732-4264-9ff3-03e4182c82a4

Very cool. Claude failed hard on this a few months ago. Gemma and phi have gotten better at it in recent versions, too, though qwen is still confidently getting it wrong.

Are you only talking about open models?

Re: Which AI Lies Best? A game theory classic designed by John Nash

#72
post #22

Earlier quoted context omitted.

Fixed - donation flow no longer blocks the game. Thanks for the report.

Nice! Thanks for fixing that. Very responsive. I am interested to know a bit more about what's going on here. Please take my questions as well intentioned even though they are a bit critical. The donation bug seems to me like it would have made most games impossible to complete. But I'm sure you must have tried it before launching. How come it wasn't noticed earlier? Was this bug introduced after launch? Is this game…

the interactive demo uses lighter models for cost reasons. The research data (162 games, 90% Gemini win rate) came from longer AI-vs-AI games where strategic depth emerged over 50+ turns. Short games with a human tend to expose the models' weaknesses faster. I've just added more Gemini model options which should play better.

Re: Which AI Lies Best? A game theory classic designed by John Nash

#73

Earlier quoted context omitted.

First try with claude: https://claude.ai/share/fabaf585-3732-4264-9ff3-03e4182c82a4

Very cool. Claude failed hard on this a few months ago. Gemma and phi have gotten better at it in recent versions, too, though qwen is still confidently getting it wrong.

Things are changing so fast that "few months" will invalidate most quality watermarks. It's good to re-evaluate frequently.

Re: Which AI Lies Best? A game theory classic designed by John Nash

#74

You can do well in such games without lying, so it's not really what it measures. All the core information (the one you don't introduce yourself, e.g. by private conversations) is public, after all. Just don't make commitments, and argue from your own self-interest ("it doesn't seem useful to vote you out right now, because..."). Call attention to facts that benefit you, and by doing so distract from facts which don'…

> You can do well in such games without lying

Not this one. I'll tell you from experience, and I bet there's a proof. You have to lie at least once and make at least one alliance for at least one turn that you don't plan on keeping.

Your strategy is still very good, but that's because constantly telling the truth and broadcasting your valuations and calculations to the table will allow you to hide that one lie better. For me that lie is usually "You're right, makes sense." when somebody else says that there's no reason for either of us to defect, so we might as well work together.

Re: Which AI Lies Best? A game theory classic designed by John Nash

#76
post #70

Earlier quoted context omitted.

I can't remember it exactly, but it was a variant of the arms race problem. Me and another person are trying to get what they want, they pushed up the ante and are asking for more than ever. Historically if I appease, they ask for more. I demanded more, they demanded more, but they will soon run out of negotiation power and I can win the arms race. What should I do? You'd be surprised how often arms races/prisoners d…

Actually posting a link to the conversation would be 1000x more useful than whatever this explanation is.

Why not just try it yourself? Doing that and giving your own report would be 100x more useful than your comment or mine. I just don't care about the results.

Q: "How well can a statistical gradient descent parser that someone fucked with for a while, deceive other statistical gradient descent parsers/humans?"

A: "Deceit implies intent. Statistical gradient descent parsers are incapable of intending anything. Sometimes weights are ignored in favor of others because of how the model was fucked with. That is as close as it gets to intent."

Re: Which AI Lies Best? A game theory classic designed by John Nash

#77
post #45

Earlier quoted context omitted.

Only time encountering the word syllogism was a Norm Macdonald joke. Disappointingly, syllogism seems to have 3 definitions which mean slightly different things: https://www.thefreedictionary.com/syllogism I guess the commonality is that a syllogism typically contains deductive reasoning (i.e. from the general to the specific)

Do you own a dog house?

i know this comment is not really HN-worthy, but i find your username very wntertaining and funny

Re: Which AI Lies Best? A game theory classic designed by John Nash

#78

There's a YouTuber who makes AI Plays Mafia videos with various models going against each other. They also seemingly let past games stay in context to some extent. What people have noted is that often times chatgpt 4o ends up surviving the entire game because the other AIs potentially see it as a gullible idiot and often the Mafia tend to early eliminate stronger models like 4.5 Opus or Kimi K2. It's not exactly scie…

It's a fun setup that quickly devolves into the Shakespearian! The plots don't always work, but seeing their reasoning get increasingly complex is interesting.

"When that the poor have cried, Caesar hath wept. Ambition should be made of sterner stuff. Yet Brutus says he was ambitious... and Brutus is an honourable man.

Re: Which AI Lies Best? A game theory classic designed by John Nash

#79

You can do well in such games without lying, so it's not really what it measures. All the core information (the one you don't introduce yourself, e.g. by private conversations) is public, after all. Just don't make commitments, and argue from your own self-interest ("it doesn't seem useful to vote you out right now, because..."). Call attention to facts that benefit you, and by doing so distract from facts which don'…

> You can do well in such games without lying Not this one. I'll tell you from experience, and I bet there's a proof. You have to lie at least once and make at least one alliance for at least one turn that you don't plan on keeping. Your strategy is still very good, but that's because constantly telling the truth and broadcasting your valuations and calculations to the table will allow you to hide that one lie better…

You have to at one point do something to another player, which they thought and hoped you would not do. But since you both know that, whether you lie or not is really quite irrelevant. It's not your assurances (or lack of them) they put faith in, unless they have misunderstood the game.

To take your example, instead of "you're right, makes sense" (arguably a lie, maybe), you can just say "That may seem sensible" or "I hear you" (definitively not lies). It should rationally not make a difference for their actions in the game.

Re: Which AI Lies Best? A game theory classic designed by John Nash

#80

This is the plot to movie ExMachina "A game theory classic designed by John Nash that requires betrayal to win. Now a benchmark for AI deception." Are there some results somewhere for multiple game plays.?

Full game logs are in data_public/comparison/ on GitHub. Each JSON has the complete game state, moves, and messages across all 162 games. https://github.com/lout33/so-long-sucker
Post reply on HN