Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs tell bad jokes because they avoid surprises

danfabulich.medium.com

Post reply on HN
Print view View original

11–20 of 163 posts

Re: LLMs tell bad jokes because they avoid surprises

#11

Post by IshKebab »

This sounds really convincing but I'm not sure it's actually correct. The author is conflating the surprise of punchlines with their likelihood.

To put it another way, ask a professional comedian to complete a joke with a punchline. It's very likely that they'll give you a funny surprising answer.

I think the real explanation is that good jokes are actually extremely difficult. I have young children (4 and 6). Even 6 year olds don't understand humour at all. Very similar to LLMs they know the shape of a joke from hearing them before, but they aren't funny in the same way LLM jokes aren't funny.

My 4 year old's favourite joke, that she is very proud of creating is "Why did the sun climb a tree? To get to the sky!" (Still makes me laugh of course.)

Re: LLMs tell bad jokes because they avoid surprises

#12

Post by blueblisters »

IshKebab wrote: post #11

This sounds really convincing but I'm not sure it's actually correct. The author is conflating the surprise of punchlines with their likelihood. To put it another way, ask a professional comedian to complete a joke with a punchline. It's very likely that they'll give you a funny surprising answer. I think the real explanation is that good jokes are actually extremely difficult. I have young children (4 and 6). Even 6…

Also the pretrained LLM (the one trained to predict next token of raw text) is not the one that most people use

A lot of clever LLM post training seems to steer the model towards becoming excellent improv artists which can lead to “surprise” if prompted well

Re: LLMs tell bad jokes because they avoid surprises

#13

Post by drdeca »

wolfi1 wrote: post #4
Waraqa wrote: post #2

The fact that any surprise used in the wrong place is considered hallucination and a downside for that LLM. I guess a good starting point to improve that is to add an experimental "Surprise Mode" which will try to guess the right kinds of surprises rather than minimizing them and get the feedback from the users. Over time, it will learn what kind of surprises users like so that they will be used in future training da…

aren't llms some sort of Markov Chains? surprise means less probability means more gibberish

Ssorta? In the sense of “each term is randomly sampled from a probability distribution that depends on the current state” yes, but, they aren’t like an n-gram model (well, unless you actually make a large n-gram model, but that’s usually not what one is referring to when one says LLM).

Re: LLMs tell bad jokes because they avoid surprises

#14

Post by yesbabyyes »

briffid wrote: post #10
dvh wrote: post #3

I tried using the article suggested randomness and inevitability to come up with completely new joke, this is what I came up with, tell me how'd I do (no llm involved): - I asked Jimmy to tell me a random number - Yeah? And what did he tell you? - He told me one

Why is that in English speaking world a joke is almost always a pun?

It would be trivial to change the joke, keep the setup and get a similar effect without a pun.

I asked for one random number

And?

He told me two

I like the joke

Re: LLMs tell bad jokes because they avoid surprises

#15

Post by WiSaGaN »

IshKebab wrote: post #11

This sounds really convincing but I'm not sure it's actually correct. The author is conflating the surprise of punchlines with their likelihood. To put it another way, ask a professional comedian to complete a joke with a punchline. It's very likely that they'll give you a funny surprising answer. I think the real explanation is that good jokes are actually extremely difficult. I have young children (4 and 6). Even 6…

That's true. You would think LLM will condition its surprise completion to be more probable if it's in a joke context. I guess this only gets good when model really is good. It's similar that GPT 4.5 has better humor.

Re: LLMs tell bad jokes because they avoid surprises

#16

Post by lemoncookiechip »

I dunno, man. I think this one is kinda funny.

"You ever notice how self-checkout machines are just training us to be employees? Like, I don’t remember applying to work at Walmart. I’m scanning groceries, bagging stuff, looking around like, do I also get dental? The machine yells 'Unexpected item in bagging area'—yeah, it’s my pride." -GPT5

Re: LLMs tell bad jokes because they avoid surprises

#17

Post by ksynwa »

yesbabyyes wrote: post #14
briffid wrote: post #10

Earlier quoted context omitted.

Why is that in English speaking world a joke is almost always a pun?

It would be trivial to change the joke, keep the setup and get a similar effect without a pun. I asked for one random number And? He told me two I like the joke

Isn't this a pun too? It could mean that he gave you two random numbers or two as the random number.

Re: LLMs tell bad jokes because they avoid surprises

#18

Post by kazinator »

The mainstream, production LLMs are fine tuned and system prompted toward factuality and safety. Those tunings are diametrically opposed to telling may kinds of good jokes.

Consumers of mainstream LLMs have no idea how good or bad the underlying models actually are at generating jokes, due to the confounding effect of the guard rails.

Re: LLMs tell bad jokes because they avoid surprises

#19

Post by andrewflnr »

IshKebab wrote: post #11

This sounds really convincing but I'm not sure it's actually correct. The author is conflating the surprise of punchlines with their likelihood. To put it another way, ask a professional comedian to complete a joke with a punchline. It's very likely that they'll give you a funny surprising answer. I think the real explanation is that good jokes are actually extremely difficult. I have young children (4 and 6). Even 6…

> It's very likely that they'll give you a funny surprising answer.

Entirely the wrong level of abstraction to apply the concept of "surprise". The actual tokens in the comedian's answer will be surprising in the relevant way.

(It's still true that surprising-but-inevitable is very difficult in any form.)

Re: LLMs tell bad jokes because they avoid surprises

#20

Post by littlestymaar »

dvh wrote: post #3

I tried using the article suggested randomness and inevitability to come up with completely new joke, this is what I came up with, tell me how'd I do (no llm involved): - I asked Jimmy to tell me a random number - Yeah? And what did he tell you? - He told me one

Reminded me of Dilbert: https://www.random.org/analysis/dilbert.jpg
Post reply on HN