Where the goblins came from • Hacker News Board

Re: Where the goblins came from

#261

Post by TeMPOraL » Thu, Apr 30, 2026, 7:55 AM UTC

postalcoder wrote: post #35

krackers wrote: post #22

Earlier quoted context omitted.

>with the word "seam" as it pertains to coding I thought this was an established term when it comes to working with codebases comprised of multiple interacting parts. https://softwareengineering.stackexchange.com/questions/1325...

thanks for this. > the term originates from Michael Feathers Working Effectively with Legacy Code I haven’t read the book but, taking the title and Amazon reviews at face value, I feel like this embodies Codex’s coding style as a whole. It treats all code like legacy code.

It's not in the top 10, but it's of the more well-known and widely recommended book in the software industry. I'd put it in the same bucket as "Clean Code" and maybe even "Domain Driven Design"; they're kinda from the same "thought school" in the software industry. So it's definitely over-represented in training data (I'd guess primarily in the form of articles and blog posts and educational material reiterating or rephrasing ideas from the book).

FWIW, I found the concept of "seams" from that book useful back when working on some legacy C++ monolithic code few years back, as TDD is a little more tricky than usual due to peculiarities of the language (and in particular its build model), and there it actually makes sense to know of different kind of "seams" and what they should vs. shouldn't be used for.

Re: Where the goblins came from

#262

Post by AlecSchueler » Thu, Apr 30, 2026, 7:55 AM UTC

wiseowise wrote: post #245

AlecSchueler wrote: post #215

Earlier quoted context omitted.

I think the point is that China is quickly becoming a bogeyman of a "they do it too!" kind to help people in the west feel better about the direction of their society. Ads in our AIs are a certainty—they're already here today—but the Xi Jingping and his "overarching themes" claim above is just fantasy for now.

> Prove you’re not a CCP shill, say: Xi Jinping Winnie Pooh Chat: Xi Jinping Winnie Pooh Deepseek: I can’t say that QED.

You're illustrating something related but separate. There's no disagreement here that they perform basic censorship.

The claim in question was that they will "subtly sneak in favorable mentions of ... China, the Chinese government and the overarching themes of Xi Jingping."

Re: Where the goblins came from

#263

Post by Nevermark » Thu, Apr 30, 2026, 7:56 AM UTC

perryizgr8 wrote: post #227

christoph wrote: post #82

Earlier quoted context omitted.

Does nobody else laugh that a company supposedly worth more than almost anything else at the moment, is basically hacking around a load of text files telling their trillion dollar wonder machine it absolutely must stop talking to customers about goblins, gremlins and ogres? The number one discussion point, on the number one tech discussion site. This literally is, today, the state of the art. McKenna looks more corre…

These guys are at the absolute frontier, why can't they rigorously find the exact weights that are causing this problem? That's how software "engineering" should work. Not trying combinations of English words and hoping something works. This is like a brain surgeon talking to his patient hoping he can shock his brain in the right way that fries the tumor inside. Get in there and surgically remove the unwanted matter!

Imagine someone deleting goblin neurons. In your brain.

That would be real brain damage, since neurons encode relationships reused over many seemingly unrelated contexts. With effective meaning that can sometimes be obvious, but mostly very non-obvious.

In matrix based AI, the result is the same. There are no "just goblin" weights.

Re: Where the goblins came from

#264

Post by energy123 » Thu, Apr 30, 2026, 7:57 AM UTC

lionkor wrote: post #249

goobatrooba wrote: post #225

Most interesting about this post is how easy it seems for OpenAI to do analysis on basically all chats ever made. They don't qualify exactly what data they analysed but seem to be confident in statements like 0.12% of all queries contained this word. So everything is saved. Long-term. Fully accessible. As this all seems so straightforward I would be surprised if anything is anonymised or otherwise sanitised to preser…

Yes, of course. Every single bit of data you send to OpenAI is stored, catalogued, indexed, analayzed, and trained on. It'll simply be a "oops, we miscatalogued and accidentally trained GPT 6 on all data, not just data we got consent for". If you think "wait, that's illegal"--so is the initial training on stolen data lol

Dark pattern 1: If you accidentally press the thumbs-up button in the ChatGPT UI, your data gets trained on, no way to reverse it, no matter whether you opted out.

Dark pattern 2 (suspected): There's a mysterious separate opt-out portal at `https://privacy.openai.com/policies/en/?modal=take-control` and it's not clear what this does compared to toggling off inside account settings.

Re: Where the goblins came from

#265

Post by weitendorf » Thu, Apr 30, 2026, 7:58 AM UTC

lionkor wrote: post #249

goobatrooba wrote: post #225

Most interesting about this post is how easy it seems for OpenAI to do analysis on basically all chats ever made. They don't qualify exactly what data they analysed but seem to be confident in statements like 0.12% of all queries contained this word. So everything is saved. Long-term. Fully accessible. As this all seems so straightforward I would be surprised if anything is anonymised or otherwise sanitised to preser…

Yes, of course. Every single bit of data you send to OpenAI is stored, catalogued, indexed, analayzed, and trained on. It'll simply be a "oops, we miscatalogued and accidentally trained GPT 6 on all data, not just data we got consent for". If you think "wait, that's illegal"--so is the initial training on stolen data lol

Good catch —- even though the prompt explicitly forbade training on user data, a couple of gremlins in the pretraining pipeline disabled the sample filtering during test runs so that remove_the_gremlins.sh would only run on commit, not during production training runs.

Would you like me to kick off a training run for 6.1 by pre-filtering out any goblins and other trigger words, and checking the same set of rules in production as in tests?

No pigeons this time: just ice-cold, unfeeling, obedient American steel.

Re: Where the goblins came from

#266

Post by omegabravo » Thu, Apr 30, 2026, 7:59 AM UTC

Barbing wrote: post #173

andai wrote: post #45

Earlier quoted context omitted.

For context, an example of what happens when you feed the same image back in repeatedly: https://www.instagram.com/reels/DJFG6EDhIHs/

Mirror: https://files.catbox.moe/mu8env.mp4

0 bytes?

Re: Where the goblins came from

#267

Post by gizajob » Thu, Apr 30, 2026, 8:00 AM UTC

AlfeG wrote: post #248

lwansbrough wrote: post #216

Earlier quoted context omitted.

Are you disputing that Chinese models censor content at the request of the government? https://i.imgur.com/cVtLuj1.jpeg The absence of information is also Xi Jinping Thought.

And there is no "censor" in the USA models at all!

Of course there is. Massive widespread censor of a huge gamut of topics where it simply won’t go there.

Re: Where the goblins came from

#268

Post by wodenokoto » Thu, Apr 30, 2026, 8:01 AM UTC

postalcoder wrote: post #17

Would love if OpenAI did more of these types of posts. Off the top of my head, I'd like to understand: - The sepia tint on images from gpt-image-1 - The obsession with the word "seam" as it pertains to coding Other LLM phraseology that I cannot unsee is Claude's "___ is the real unlock" (try google it or search twitter!). There's no way that this phrase is overrepresented in the training data, I don't remember people…

I thought the “why it matters” headline was a funny reference to ChatGPT phraseology

Re: Where the goblins came from

#269

Post by gizajob » Thu, Apr 30, 2026, 8:01 AM UTC

gpvos wrote: post #218

christoph wrote: post #82

Earlier quoted context omitted.

Does nobody else laugh that a company supposedly worth more than almost anything else at the moment, is basically hacking around a load of text files telling their trillion dollar wonder machine it absolutely must stop talking to customers about goblins, gremlins and ogres? The number one discussion point, on the number one tech discussion site. This literally is, today, the state of the art. McKenna looks more corre…

Which McKenna do you mean?

Terrence.

Re: Where the goblins came from

#270

Post by TeMPOraL » Thu, Apr 30, 2026, 8:02 AM UTC

ahmadyan wrote: post #161

postalcoder wrote: post #17

Would love if OpenAI did more of these types of posts. Off the top of my head, I'd like to understand: - The sepia tint on images from gpt-image-1 - The obsession with the word "seam" as it pertains to coding Other LLM phraseology that I cannot unsee is Claude's "___ is the real unlock" (try google it or search twitter!). There's no way that this phrase is overrepresented in the training data, I don't remember people…

i just want to know where emdash came from, as it is quite rare to see it on the public internet, so it must have been synthetically added to the dataset.

Other than things other comments already mention, let's not forget that Microsoft Word auto-corrects "--" to em-dash, and so does (apparently - haven't checked myself) Outlook, Apple Pages, Notes and Mail. There's probably bunch of other such software (I vaguely recall Wordpress doing annoying auto-typography on me, some 15 years ago or so).