Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Where the goblins came from

openai.com

Post reply on HN
Print view View original

201–210 of 699 posts

Re: Where the goblins came from

#201

Post by yard2010 »

dakolli wrote: post #9
JoshTriplett wrote: post #5

A plausible theory I've seen going around: https://x.com/QiaochuYuan/status/2049307867359162460

It is a stateless text / pixel auto-complete it has no references of self, stop spreading this bs.

Imagine people would just click words on iOS auto complete mistaking this for intelligence:

"I think the problem is that when you don't have to be perfect for me that's why I'm asking you to do it but I would love to see you guys too busy to get the kids to the park and the trekkers the same time as the terrorists."

How do you like this theory?

Re: Where the goblins came from

#202

Post by mentalgear »

ollin wrote: post #15

For context, two days ago some users [1] discovered this sentence reiterated throughout the codex 5.5 system prompt [2]: > Never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely and unambiguously relevant to the user's query. [1] https://x.com/arb8020/status/2048958391637401718 [2] https://github.com/openai/codex/blob/main/codex-rs/models-ma...

Apparently there is a mushroom that makes most people have the same hallucinations of "little people" or similar fantasy figures. Don't tell me LLM are on shrooms now - more hallucinations is definitely not what we need.

> Scientists call them “lilliputian hallucinations,” a rare phenomenon involving miniature human or fantasy figures

https://news.ycombinator.com/item?id=47918657

Re: Where the goblins came from

#203

Post by krackers »

layer8 wrote: post #194
krackers wrote: post #22

Earlier quoted context omitted.

>with the word "seam" as it pertains to coding I thought this was an established term when it comes to working with codebases comprised of multiple interacting parts. https://softwareengineering.stackexchange.com/questions/1325...

No, it’s not an established term outside the mentioned books, beyond the generic meaning of the word.

I have frequently encountered the term in the context of unit testing and dependency injection.

Other references (and all predate chatgpt):

>Seams are places in your code where you can plug in different functionality

>Art of Unit Testing, 2nd edition page 54

(https://blog.sasworkshops.com/unit-testing-and-seams/)

>With the help of a technique called creating a seam, or subclass and override we can make almost every piece of code testable.

https://www.hodler.co/2015/12/07/testing-java-legacy-code-wi...

> seam; a point in the code where I can write tests or make a change to enable testing

https://danlimerick.wordpress.com/2012/06/11/breaking-hidden...

Maybe it all ultimately traces back to the book mentioned before, but I don't believe it's an obscure term in the circles of java-y enterprise code/DI. In fact the only reason I know the term is because that's how dependency injection was first defined to me (every place you inject introduces a "seam" between the class being injected and the class you're injecting into, which allows for easy testing). I can't remember where exactly I encountered that definition though.

Re: Where the goblins came from

#204

Post by qingcharles »

heavyset_go wrote: post #124
tdeck wrote: post #96

Earlier quoted context omitted.

Is this the "prompt engineering" that I keep hearing will be an indispensable job skill for software engineers in the AI-driven future? I had better start learning or I'll be replaced by someone who has.

If you aren't telling your computer to ignore goblins, you're going to be left behind.

I'm goblinmaxxing myself.

Re: Where the goblins came from

#206

Post by frm88 »

emsign wrote: post #190
christoph wrote: post #82

Earlier quoted context omitted.

Does nobody else laugh that a company supposedly worth more than almost anything else at the moment, is basically hacking around a load of text files telling their trillion dollar wonder machine it absolutely must stop talking to customers about goblins, gremlins and ogres? The number one discussion point, on the number one tech discussion site. This literally is, today, the state of the art. McKenna looks more corre…

Exactly my first thought. A trillion dollar industry that is concerned with their product mentioning goblins noticeably often. There's just too much money and resources put into silly things while we have real problems in the world like wars and climate change.

This, very much. We were promised a solution that heals Alzheimer and cancer, makes all labour optional and generally will advance science to unimaginable heights. Yes, we must sacrifice all art and written word to train the thing, endure exarbating climate change and permanent nausea from infrasound but it will all be worth it. 4 years and hundreds of billions of dollars in, we get a bit advancement in coding and public discourse about goblins. Oh, and intelligent weaponry. At this point I think the priorities are clear.

Re: Where the goblins came from

#208

Post by brookst »

zozbot234 wrote: post #131
christoph wrote: post #82

Earlier quoted context omitted.

Does nobody else laugh that a company supposedly worth more than almost anything else at the moment, is basically hacking around a load of text files telling their trillion dollar wonder machine it absolutely must stop talking to customers about goblins, gremlins and ogres? The number one discussion point, on the number one tech discussion site. This literally is, today, the state of the art. McKenna looks more corre…

Spoiler: future versions of mainstream AIs will be fine tuned in the exact same way to subtly sneak in favorable mentions of sponsored products as part of their answers. And Chinese open-weight AIs will do the exact same thing, only about China, the Chinese government and the overarching themes of Xi Jinping Thought.

I’m very skeptical that training is the right way to insert ads.

Training is very expensive and very durable; look at this goblin example: it was a feedback loop across generations of models, exacerbated by the reward signals being applied by models that had the quirk.

How does that work for ads? Coke pays to be the preferred soda… forever? There’s no realtime bidding, no regional ad sales, no contextual sales?

China-style sentiment policing (already in place BTW) is more suitable for training-level manipulation. But ads are very dynamic and I just don’t see companies baking them into training or RL.

Re: Where the goblins came from

#209

Post by dyauspitr »

postalcoder wrote: post #17

Would love if OpenAI did more of these types of posts. Off the top of my head, I'd like to understand: - The sepia tint on images from gpt-image-1 - The obsession with the word "seam" as it pertains to coding Other LLM phraseology that I cannot unsee is Claude's "___ is the real unlock" (try google it or search twitter!). There's no way that this phrase is overrepresented in the training data, I don't remember people…

“I’ve got the shape of it now”

Re: Where the goblins came from

#210

Post by jchw »

jruz wrote: post #195
zozbot234 wrote: post #131

Earlier quoted context omitted.

Spoiler: future versions of mainstream AIs will be fine tuned in the exact same way to subtly sneak in favorable mentions of sponsored products as part of their answers. And Chinese open-weight AIs will do the exact same thing, only about China, the Chinese government and the overarching themes of Xi Jinping Thought.

Is this Xi Jinping with us in the room right now?

Are you implying that Xi Jinping is not real? I'm pretty sure that's not how that snowclone works...
Post reply on HN