Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Where the goblins came from

openai.com

Post reply on HN
Print view View original

341–350 of 699 posts

Re: Where the goblins came from

#341

Post by Sharlin »

kdheiwns wrote: post #324
zozbot234 wrote: post #131

Earlier quoted context omitted.

Spoiler: future versions of mainstream AIs will be fine tuned in the exact same way to subtly sneak in favorable mentions of sponsored products as part of their answers. And Chinese open-weight AIs will do the exact same thing, only about China, the Chinese government and the overarching themes of Xi Jinping Thought.

American AIs only do this and promote American values. Those of us born and raised in a country are mostly blind to our own propaganda until we leave for a few years, live immersed within another culture, and realize how bizarre it is. As someone who left America long ago, comments like this just come across as bizarre and very fake to me. A few years ago I might've thought "whoa dude that's deep" But basically, Chin…

That’s a rather weird and non-sequitur take of what the GP said.

Re: Where the goblins came from

#342

Post by creationcomplex »

antonvs wrote: post #306
libraryofbabel wrote: post #222

Earlier quoted context omitted.

It's interesting that some people are responding to your comment as if this proves that AI is a sham or a joke. But I don't think that's what you're saying at all with your reference to Terence McKenna: this is a serious thing we're talking about here! These models are alien intelligences that could occupy an unimaginably vast space of possibilities (there are trillions of weights inside them), but which have been RL…

> and also it's deeply weird and secretly obsessed with goblins and gremlins. Only because its makers insist on trying to give them "personality".

This is the eye opener - they're degrading the model for novelties.

Re: Where the goblins came from

#344

Post by kindkang2024 »

nomilk wrote: post #4

> We unknowingly gave particularly high rewards for metaphors with creatures. I recall a math instructor who would occasionally refer to variables (usually represented by intimidating greek letters) as "this guy". Weirdly, the casual anthropomorphism made the math seem more approachable. Perhaps 'metaphors with creatures' has a similar effect i.e. makes a problem seem more cute/approachable. On another note, buzzword…

Show me the incentives, I'll show you the outcome.

Timeless, be it human or machine

Re: Where the goblins came from

#345

Post by Sharlin »

libraryofbabel wrote: post #222
christoph wrote: post #82

Earlier quoted context omitted.

Does nobody else laugh that a company supposedly worth more than almost anything else at the moment, is basically hacking around a load of text files telling their trillion dollar wonder machine it absolutely must stop talking to customers about goblins, gremlins and ogres? The number one discussion point, on the number one tech discussion site. This literally is, today, the state of the art. McKenna looks more corre…

It's interesting that some people are responding to your comment as if this proves that AI is a sham or a joke. But I don't think that's what you're saying at all with your reference to Terence McKenna: this is a serious thing we're talking about here! These models are alien intelligences that could occupy an unimaginably vast space of possibilities (there are trillions of weights inside them), but which have been RL…

…But this goblin thing was a direct result of accidentally creating a positive feedback loop in RL to make the model more human-like, nothing about unintentionally surfacing an aspect of Cthulhu from the depths despite attempts to keep the model humanlike. This is not a quirk of the base model but simply a case of reinforcement learning being, well, reinforcing.

Re: Where the goblins came from

#346

Post by fragmede »

cultofmetatron wrote: post #331
AlfeG wrote: post #248

Earlier quoted context omitted.

And there is no "censor" in the USA models at all!

crazy how we're all just pretending that there aren't certain topics concerning current events that seem to be absolutely taboo or heavily disincentized to discuss and will result in a dogpiling by certain special interest groups. we all know who they are and yet we all tacitly accept it.

Current events? Ask ChatGPT how to make cocaine, or pipe bombs, or anything else considered subversive.

Re: Where the goblins came from

#347

Post by walthamstow »

gizajob wrote: post #272
tudorpavel wrote: post #72

Earlier quoted context omitted.

The one phrase that irks me as overly dramatic and both GPT and Claude use it a lot is "__ is the real smoking gun!" I'm a non-native English speaker, so maybe it's a really common idiom to use when debugging?

I’m a British English speaker and find the use of cliched American idioms really quite disgusting. Don’t want to think about about ballparks, home runs, smoking guns, going all in, touchdowns or hitting it out the park.

In my user instructions I always have a point to "always use British English" which seems to reduce Americanisms. I am yet to see Claude give me a "back of the net!" though, sadly.

Re: Where the goblins came from

#348

Post by latexr »

christoph wrote: post #82
ollin wrote: post #15

For context, two days ago some users [1] discovered this sentence reiterated throughout the codex 5.5 system prompt [2]: > Never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely and unambiguously relevant to the user's query. [1] https://x.com/arb8020/status/2048958391637401718 [2] https://github.com/openai/codex/blob/main/codex-rs/models-ma...

Does nobody else laugh that a company supposedly worth more than almost anything else at the moment, is basically hacking around a load of text files telling their trillion dollar wonder machine it absolutely must stop talking to customers about goblins, gremlins and ogres? The number one discussion point, on the number one tech discussion site. This literally is, today, the state of the art. McKenna looks more corre…

> Does nobody else laugh (…)

To an extent, yes. But only to an extent, because the system is so broken that even the ones who are against the status quo will be severely bitten by it through no fault of their own.

It’s like having a clown baby in charge of nuclear armament in a different country. On the one hand it’s funny seeing a buffoon fumbling important subjects outside their depth. It could make for great fictional TV. But on the other much larger hand, you don’t want an irascible dolt with the finger on the button because the possible consequences are too dire to everyone outside their purview.

Re: Where the goblins came from

#349

Post by Dilettante_ »

bigyabai wrote: post #212
jruz wrote: post #195

Earlier quoted context omitted.

Is this Xi Jinping with us in the room right now?

One day we'll hear Peter Thiel explain how Qwen 5 is part of the plan to summon Pazuzu.

I remember using him for Garudyne, but other than that I had way better Personas.

Re: Where the goblins came from

#350

Post by ProllyInfamous »

mentalgear wrote: post #202
ollin wrote: post #15

For context, two days ago some users [1] discovered this sentence reiterated throughout the codex 5.5 system prompt [2]: > Never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely and unambiguously relevant to the user's query. [1] https://x.com/arb8020/status/2048958391637401718 [2] https://github.com/openai/codex/blob/main/codex-rs/models-ma...

Apparently there is a mushroom that makes most people have the same hallucinations of "little people" or similar fantasy figures. Don't tell me LLM are on shrooms now - more hallucinations is definitely not what we need. > Scientists call them “lilliputian hallucinations,” a rare phenomenon involving miniature human or fantasy figures https://news.ycombinator.com/item?id=47918657

>there is a mushroom

Ketamine == angels

DMT == little shadow elves

Salvia == devils

...or so I've heard.

Post reply on HN