Where the goblins came from • Hacker News Board

Re: Where the goblins came from

#681

Post by nearbuy » Fri, May 01, 2026, 5:37 PM UTC

airstrike wrote: post #665

nearbuy wrote: post #622

Earlier quoted context omitted.

Just to clarify, it's not the prompt voodoo that caused the affinity for goblins. It's the reward. They rewarded it for mentioning goblins when set to Nerdy, and it's still the same model as the other personalities, so the effects can carry over.

Makes sense, but I don't know why they'd let said prompt voodoo touch RL. I'm OK with prompting to get the model to, I don't know, write better Rust or build Excel spreadsheets. I am less OK with making it "quirky" or having some "personality" in a way that becomes ingrained in the model for everyone else TL;DR the cringe nerdy shit should be (optionally) switched on at inference, not as part of RL

They do it because training different personalities is more effective than just changing the system prompt. Ever try asking ChatGPT to adopt a specific personality in a prompt? Its standard style bleeds through.

As the article says, the personalities weren't supposed to affect other personalities. OpenAI was as surprised by the goblins as you are. Training can be tricky.

Re: Where the goblins came from

#682

Post by gspetr » Fri, May 01, 2026, 6:51 PM UTC

Silamoth wrote: post #517

krackers wrote: post #203

Earlier quoted context omitted.

I have frequently encountered the term in the context of unit testing and dependency injection. Other references (and all predate chatgpt): >Seams are places in your code where you can plug in different functionality >Art of Unit Testing, 2nd edition page 54 ( https://blog.sasworkshops.com/unit-testing-and-seams/ ) >With the help of a technique called creating a seam, or subclass and override we can make almost every…

For what it’s worth, there are many areas of programming where dependency injection is almost never used. Game dev, data science, and embedded systems, for example, rarely use dependency injection. It’s definitely most common in enterprise Java code and less common in Python, C, or C++. And even then, not everyone uses the term “seam”.

[flagged]

Re: Where the goblins came from

#683

Post by theowaway » Fri, May 01, 2026, 7:43 PM UTC

fud101 wrote: post #664

theowaway wrote: post #402

Earlier quoted context omitted.

I think you need to go outside and touch some grass

He's clearly mimicking one of the clankers.

undoubtedly. But my statement still stands

Re: Where the goblins came from

#684

Post by lwansbrough » Fri, May 01, 2026, 11:46 PM UTC

fragmede wrote: post #652

lwansbrough wrote: post #641

Earlier quoted context omitted.

Do you have any actual examples of political history being actively censored by western models? Or are we just doing hypotheticals for fun?

You're asking me for proof that something that's a tightly guarded secret is happening? I don't work at OpenAI or anything so I don't know why you think I'd have that. As far as doing it for fun, no, this is a serious matter to me, is it not for you? Still, if you ask ChatGPT or Claude details on what's going on in the western bank, Israel and Gaza, there's a specific viewpoint being pushed. I am not remotely qualifi…

I was able to pull up an example of a Chinese model doing censorship in 2 seconds. So there is clearly a difference in the type of censorship happening if it’s harder than that for you to prove.

Your example is already under dispute by actual humans. Expecting non-AGI to get it right is not realistic.

Re: Where the goblins came from

#685

Post by HanyouHottie » Sat, May 02, 2026, 3:08 AM UTC

At the time, the prevalence of goblins did not look especially alarming. A few months later, the goblins came back to haunt us in a much more specific and reproducible form.

-OpenAI

Crazy timeline we're living in.

Re: Where the goblins came from

#686

Post by ashtonshears » Sat, May 02, 2026, 4:26 AM UTC

idiotsecant wrote: post #488

ashtonshears wrote: post #475

Earlier quoted context omitted.

Poor correlation comparing physical material to computer technology

Why

Computer technology has many significant unique factors compared to other technologies

Re: Where the goblins came from

#687

Post by irishcoffee » Sat, May 02, 2026, 10:22 AM UTC

wincy wrote: post #591

irishcoffee wrote: post #478

Earlier quoted context omitted.

Asbestos, lead paint, cigarettes, heroin(perscribed generously for basically whatever the doc felt like), "Radithor" (patent medicine containing radium-226 and 228, marketed as a "perpetual sunshine" energy tonic and cure for over 150 diseases), bloodletting, mercury treatments for syphilis, tobacco smoke enemas (yep that was a real thing), milk-based blood transfusions. Didn't understand those either and used the fu…

Smoking cigarettes didn’t really matter for as long as we were regularly burning wood for fuel. Turns out just burning pretty much anything and breathing in the particles is really bad for you. Makes sense we didn’t realize it was bad until we stopped burning logs and coal for home heating and cooking.

I didn't mention smoking cigarettes at all. I said people literally blew smoke up their ass. Huh, I wonder if that's where the saying came from, now that I think about it.

Re: Where the goblins came from

#688

Post by chongli » Sat, May 02, 2026, 10:55 AM UTC

jychang wrote: post #672

chongli wrote: post #668

Earlier quoted context omitted.

> people within the species would not easily recognize it [Citation Needed] Sorry, but I had to. There's easy counterexamples of true, species-wide biases that we're fully aware of. Optical illusions, cognitive biases, cultural universals (community-sanctioned relationships/marriage, inheritance, ceremonial treatment of the dead). What we don't have are universal biases towards believing specific facts or stories.

None of those things are easily recognized though. They're not universals. A term like "cognitive biases" generally require a college level education. If you go to a tribe in the middle of the rainforest, would they be able to explain those concepts? Of course not. Plus, I already gave an example of a species wide bias at the end of the comment- phone addiction for kids. I'm clearly not saying it's impossible for a h…

You’ve moved the goalposts.

You’ve gone from “people within the species not being able to easily recognize a bias” to “people universally recognizing that bias, even with no education or contact with the rest of civilization.”

That’s silly, and something I’d never argue for. To me, something is easy for humans to recognize if a 19th century scientist could discover it. We are a social and cultural species. Culture is how we learn anything over the long run.

Re: Where the goblins came from

#689

Post by sarchertech » Sat, May 02, 2026, 2:19 PM UTC

red-iron-pine wrote: post #677

sarchertech wrote: post #601

Earlier quoted context omitted.

Cigarettes actually are uniquely bad when it comes to lung cancer. Lung cancer was very rare in 1900 and before when everyone was still burning wood or coal for warmth and cooking. Lung cancer rates didn’t take off until cigarette popularity exploded after WWI. Chewing tobacco also causes mouth cancer, so there’s more to it than just inhaling byproducts of combustion.

1) people smoked a lot more in the post WWI econ boom 2) additives or even just paper compound the negative effects of the smoke on the lungs. like, firefighters - who usually have physical fitness requirements and don't smoke - see rates of lung cancer similar to moderate smokers, simply due to the higher volume of particulate and chems hitting their lungs. it is dose-dependant, and firefighers who see more fires se…

I’m unsure where your firefighters have the same lung cancer risk as moderate smokers information comes from, but it’s wrong.

https://pmc.ncbi.nlm.nih.gov/articles/PMC7063017/

Here’s an meta-analysis of 49 studies that shows no increase in lung cancer.

And of course it’s dose dependent. But newer studies show that years smoking is much more important than intensity when it comes to lung cancer risk. So smoking half a pack a day for 20 years is worse than a pack a day for 10 years.

Dry snuff comes with a 2-8x increase in oral cancer and a 10-12x increase in nasal and sinus cancer.

Tobacco is a carcinogen—even without additives. In addition to epidemiological evidence we have a plausible mechanism of action.

Alkaloids in the leaf convert into carcinogenic TSNAs during curing, aging, or drying. Tobacco plants absorbs heavy metals. And tobacco plants absorb polonium-210.

There’s a lot of misinformation and misleading interpretations out there that come from years of the tobacco industry attempting to create uncertainty. Especially with your firefighter myth, I think you might have got hold of some of it.

Re: Where the goblins came from

#690

Post by ordinarily » Sat, May 02, 2026, 3:33 PM UTC

Terr_ wrote: post #50

ordinarily wrote: post #40

Earlier quoted context omitted.

Doesn't seem that surprising or terrifying to me. Humans come equipped with a lot more internal biases (learned in a fairly similar fashion), and they're usually a lot more resistant to getting rid of them. The truly terrifying stuff never makes it out of the RLHF NDAs.

We ought to be terrified, when one adjusts for ll the use-cases people are talking about using these algorithms in. (Even if they ultimately back off, it's a lot of frothy bubble opportunity cost.) There a great many things people do which are not acceptable in our machines. Ex: I would not be comfortable flying on any airplane where the autopilot "just zones-out sometimes", even though it's a dysfunction also seen i…

I just mean ontologically, it does not surprise nor terrify me that a machine built to simulate human output also simulates the worst of us.