[flagged]
Where the goblins came from
71–80 of 699 posts
Re: Where the goblins came from
#72Would love if OpenAI did more of these types of posts. Off the top of my head, I'd like to understand: - The sepia tint on images from gpt-image-1 - The obsession with the word "seam" as it pertains to coding Other LLM phraseology that I cannot unsee is Claude's "___ is the real unlock" (try google it or search twitter!). There's no way that this phrase is overrepresented in the training data, I don't remember people…
The one phrase that irks me as overly dramatic and both GPT and Claude use it a lot is "__ is the real smoking gun!"
I'm a non-native English speaker, so maybe it's a really common idiom to use when debugging?
Re: Where the goblins came from
#73This is funny because it’s a silly topic, but I think it shows something extremely seriously wrong with llms. The goblins stand out because it’s obvious. Think of all the other crazy biases latent in every interaction that we don’t notice because it’s not as obvious. Absolutely terrifying that OpenAI is just tossing around that such subtle training biases were hard enough to contain it had to be added to system promp…
We do not have the complete picture.
Re: Where the goblins came from
#74This is funny because it’s a silly topic, but I think it shows something extremely seriously wrong with llms. The goblins stand out because it’s obvious. Think of all the other crazy biases latent in every interaction that we don’t notice because it’s not as obvious. Absolutely terrifying that OpenAI is just tossing around that such subtle training biases were hard enough to contain it had to be added to system promp…
I think it's extraordinarily telling that people are capable of being reflexively pessimistic in response to the goblin plague. It's like something Zitron would do. This story is wonderful .
I feel at least partially responsible. I would often instruct agents to "stop being a goblin". I really enjoyed this story too, though.
Re: Where the goblins came from
#75Earlier quoted context omitted.
> If we all had the exact same bias then it would be a huge problem. And may I introduce you to "groupthink" :))
Now imagine that every opinion you have is automatically fully groupthinked and you see the difference/problem with training up a big AI model that has a hundred million users. The problem does exist when using individual humans but in a much smaller form.
> The problem does exist when using individual humans but in a much smaller form.
And may I introduce you to organized religion :)
Re: Where the goblins came from
#76Would love if OpenAI did more of these types of posts. Off the top of my head, I'd like to understand: - The sepia tint on images from gpt-image-1 - The obsession with the word "seam" as it pertains to coding Other LLM phraseology that I cannot unsee is Claude's "___ is the real unlock" (try google it or search twitter!). There's no way that this phrase is overrepresented in the training data, I don't remember people…
The one phrase that irks me as overly dramatic and both GPT and Claude use it a lot is "__ is the real smoking gun!" I'm a non-native English speaker, so maybe it's a really common idiom to use when debugging?
It probably was found in a bunch of meaningful code commit messages
Re: Where the goblins came from
#77A plausible theory I've seen going around: https://x.com/QiaochuYuan/status/2049307867359162460
I wish the blog mentioned more about why exactly training for nerdy personality rewarded mention of goblins. Since it's probably not a deterministic verifiable reward, at their level the reward model itself is another LLM. But this just pushes the issue down one layer, why did _that_ model start rewarding mentions of goblin?
> I wish the blog mentioned more about why exactly training for nerdy personality rewarded mention of goblins. Since it's probably not a deterministic verifiable reward, at their level the reward model itself is another LLM. But this just pushes the issue down one layer, why did _that_ model start rewarding mentions of goblin?
Speculation: because nerds stereotypically like sci-fi and fantasy to an unhealthy degree, and goblins, gremlins, and trolls are fantasy creatures which that stereotype should like? Then maybe goblins hit a sweet spot where it could be a problem that could sneak up on them: hitting the stereotype, but not too out of place to be immediately obnoxious.
Re: Where the goblins came from
#78A plausible theory I've seen going around: https://x.com/QiaochuYuan/status/2049307867359162460
It is a stateless text / pixel auto-complete it has no references of self, stop spreading this bs.
It has trained on vast amounts of content that contains the concept of self, of course the idea of self is emergent.
And autoregressive LLMs are not stateless.
Re: Where the goblins came from
#79Would love if OpenAI did more of these types of posts. Off the top of my head, I'd like to understand: - The sepia tint on images from gpt-image-1 - The obsession with the word "seam" as it pertains to coding Other LLM phraseology that I cannot unsee is Claude's "___ is the real unlock" (try google it or search twitter!). There's no way that this phrase is overrepresented in the training data, I don't remember people…
"shape" too, at least with gpt5.5, is coming up constantly.
Re: Where the goblins came from
#80> We unknowingly gave particularly high rewards for metaphors with creatures. I recall a math instructor who would occasionally refer to variables (usually represented by intimidating greek letters) as "this guy". Weirdly, the casual anthropomorphism made the math seem more approachable. Perhaps 'metaphors with creatures' has a similar effect i.e. makes a problem seem more cute/approachable. On another note, buzzword…
> I recall a math instructor who would occasionally refer to variables (usually represented by intimidating greek letters) as "this guy".
I also had an instructor who was doing that! This was 20 years ago, and I totally forgot about it until I have read your comment. Can’t remember the subject, maybe propositional logic? I wonder if my instructor and your instructor have picked up this habit from the same source.