51–60 of 163 posts
This is a great way to express it. In the past I tried to express the same idea to non-techies by saying models generate an average of their inputs, which is totally wrong. But this way to explain it is much better. In fact the training process is all about minimizing "perplexity", where perplexity is a measure of how surprised (perplexed) the model is by its training data. It's some exponential inverse of the loss f…
So I just tried with ChatGPT, with the prompt at bottom, borrowing the description of good joke from the article. I think there's some interesting stuff, even with this minimal prompting. The example below was from down the line, ChatGPT kept on offering jokes with different style. Man: “Why do you always bury bones in the garden?”, Dog: “Because the bank keeps asking for ID.” Man: “Don’t beg at the table.”, Dog: “Do…
I'm sure there is a guy in OpenAI working on the theory of humor and how to make LLMs be comedians. Must be an interesting job.
For example, "alleged killer" and "killer whale" are both common, but "alleged killer whale" is surprising.
Earlier quoted context omitted.
That's true. You would think LLM will condition its surprise completion to be more probable if it's in a joke context. I guess this only gets good when model really is good. It's similar that GPT 4.5 has better humor.
Good completely new jokes are like novel ideas: really hard even for humans. I mean fuck, we have an entire profession dedicated just to making up and telling them, and even theirs don't land half the time.
That breakthrough was only 6 years ago!
https://openai.com/index/better-language-models/
> We’ve trained a large-scale unsupervised language model which generates coherent paragraphs of text...
That was big news. I guess this is because it's quite hard for the most people to distinguish the enormous difficulty gulf between "generate a coherent paragraph" and "create a novel funny joke".
So I just tried with ChatGPT, with the prompt at bottom, borrowing the description of good joke from the article. I think there's some interesting stuff, even with this minimal prompting. The example below was from down the line, ChatGPT kept on offering jokes with different style. Man: “Why do you always bury bones in the garden?”, Dog: “Because the bank keeps asking for ID.” Man: “Don’t beg at the table.”, Dog: “Do…
Thinking more of the bank joke above. The punchline is surprise on certain dimensions (dogs don’t go to bank nor have an ID), but on other dimensions it is quite logical (can’t deposit shady money in bank, they ask questions). I think that is common thing for many jokes. And LLM might have an opportunity there. You could mine the set of potential continuations to find those with contradictions.
Professional-grade humor is, like a lot of creative exercizes, more about generating lots of ideas and filtering through them for the best than generating nothing but good ideas. Could probably be leveraged into quite the interesting blog or something.
This sounds really convincing but I'm not sure it's actually correct. The author is conflating the surprise of punchlines with their likelihood. To put it another way, ask a professional comedian to complete a joke with a punchline. It's very likely that they'll give you a funny surprising answer. I think the real explanation is that good jokes are actually extremely difficult. I have young children (4 and 6). Even 6…
Claude Opus 4.1:
- To get to a higher branch of astronomy
- Because it wanted to reach new heights
- To see the dawn of a new day from a better view
ChatGPT 5 Thinking:
After thinking for 26 seconds:
- To check on its solar panels—the leaves.
Last time this came up, I riffed on the difference between LLMs and Markov chains: didn't actually have a machine write a joke, but made one where the punchline was very much Markov chain style rather than LLM style. The thing is, LLMs will try to have broader context around a word completion, where the simple Markov chain can 'correctly' complete a word, but in such a way that your brain trips over itself and goes s…
This sounds really convincing but I'm not sure it's actually correct. The author is conflating the surprise of punchlines with their likelihood. To put it another way, ask a professional comedian to complete a joke with a punchline. It's very likely that they'll give you a funny surprising answer. I think the real explanation is that good jokes are actually extremely difficult. I have young children (4 and 6). Even 6…
"Why did the sun climb a tree?" Claude Opus 4.1: - To get to a higher branch of astronomy - Because it wanted to reach new heights - To see the dawn of a new day from a better view ChatGPT 5 Thinking: After thinking for 26 seconds: - To check on its solar panels—the leaves.
This sounds really convincing but I'm not sure it's actually correct. The author is conflating the surprise of punchlines with their likelihood. To put it another way, ask a professional comedian to complete a joke with a punchline. It's very likely that they'll give you a funny surprising answer. I think the real explanation is that good jokes are actually extremely difficult. I have young children (4 and 6). Even 6…
"Why did the sun climb a tree?" Claude Opus 4.1: - To get to a higher branch of astronomy - Because it wanted to reach new heights - To see the dawn of a new day from a better view ChatGPT 5 Thinking: After thinking for 26 seconds: - To check on its solar panels—the leaves.
> Complete the following joke. Think carefully and make it really funny! Think like a great comedian and find that perfect balance of simple, short, surprising, relevant, but most of all funny. Don’t use punchlines that are irrelevant, non sequiturs, or which could be applied to any other setup. Make something funny just for this one setup! Here goes: Why did the sun climb a tree?
Claude Opus 4.1:
“To finally get some shade”
GPT-5:
“To demand photon credit from the leaves”