Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Where the goblins came from

openai.com

Post reply on HN
Print view View original

281–290 of 699 posts

Re: Where the goblins came from

#281

Post by jijijijij »

tudorpavel wrote: post #72
postalcoder wrote: post #17

Would love if OpenAI did more of these types of posts. Off the top of my head, I'd like to understand: - The sepia tint on images from gpt-image-1 - The obsession with the word "seam" as it pertains to coding Other LLM phraseology that I cannot unsee is Claude's "___ is the real unlock" (try google it or search twitter!). There's no way that this phrase is overrepresented in the training data, I don't remember people…

The one phrase that irks me as overly dramatic and both GPT and Claude use it a lot is "__ is the real smoking gun!" I'm a non-native English speaker, so maybe it's a really common idiom to use when debugging?

> I'm a non-native English speaker, so maybe it's a really common idiom to use when debugging?

No. But it is something goblins say a lot.

Re: Where the goblins came from

#282

Post by TheOtherHobbes »

jruz wrote: post #195
zozbot234 wrote: post #131

Earlier quoted context omitted.

Spoiler: future versions of mainstream AIs will be fine tuned in the exact same way to subtly sneak in favorable mentions of sponsored products as part of their answers. And Chinese open-weight AIs will do the exact same thing, only about China, the Chinese government and the overarching themes of Xi Jinping Thought.

Is this Xi Jinping with us in the room right now?

It's called the Chinese Room for a reason.

Re: Where the goblins came from

#283

Post by hansmayer »

ben_w wrote: post #180
hansmayer wrote: post #174

> We unknowingly gave particularly high rewards for metaphors with creatures. From there, the goblins spread. WTF does this even mean? How the hell do you do something like this "unknowingly"? What other features are you bumping "unknowingly"? Suicide suggestions or weapon instructions come to mind. Horrible, this ship obviously has no captain!

Yes? They know, they'e always known. Why do you think they've been saying, since GPT-2, not ChatGPT even, that their LLMs needs careful study before being released?

Well obviously they have - but the press and the common folk still treat these people as some kind of geniuses, when they are obviously more similar to that junior dev using some framework without understanding its internals.

Re: Where the goblins came from

#284

Post by weitendorf »

AyanamiKaine wrote: post #255

I find it somewhat sad, too see personality changes as a bug. I dont know why but it gives me a sad feeling.

I think if you see it as weird social phases that the model lacks the self-awareness to identify as kinda embarrassing, it makes more sense.

Like if a human were going around saying “for the culture!” so much at work that they didn’t realize why telling their coworker “Oh yeah, grief counseling for the culture!” is weird coming from a white person in a serious context, it kinda makes you wonder what else they are totally oblivious about and if they even know what they’re saying actually means.

They literally need the human feedback/to learn model why some behavior is acceptable or even humorous in certain contexts but an absolute faux pas in others.

I think in the long run though we can just give people to the option to include access to human facial data/embeddings during conversations so they can pick up on body language, I think I kinda agree in a sense that direct language policing via SFT feels unnecessarily blunt and rudimentary since it doesn’t help them model the processes behind the feedback (until maybe one day some future model ends up training on the article or code and closes the loop!)

Re: Where the goblins came from

#285

Post by jijijijij »

ahmadyan wrote: post #161
postalcoder wrote: post #17

Would love if OpenAI did more of these types of posts. Off the top of my head, I'd like to understand: - The sepia tint on images from gpt-image-1 - The obsession with the word "seam" as it pertains to coding Other LLM phraseology that I cannot unsee is Claude's "___ is the real unlock" (try google it or search twitter!). There's no way that this phrase is overrepresented in the training data, I don't remember people…

i just want to know where emdash came from, as it is quite rare to see it on the public internet, so it must have been synthetically added to the dataset.

It has been rare. It's common now, even in meaningful human texts. (I know because I detest the correct usage without spaces, t looks wrong.) One of the ways AI is shaping our minds.

Re: Where the goblins came from

#287

Post by Ekaros »

chongli wrote: post #42
ninjagoo wrote: post #38

Earlier quoted context omitted.

> Absolutely terrifying that OpenAI is just tossing around that such subtle training biases were hard enough to contain it had to be added to system prompt. May I introduce you to homo sapiens , a species so vulnerable to such subtle (or otherwise) biases (and affiliations) that they had to develop elaborate and documented justice systems to contain the fallouts? :)

We’re really not that vulnerable to such things as a species, because we as individuals all have our own minds and our own sets of biases that cancel out and get lost in the noise. If we all had the exact same bias then it would be a huge problem.

Doesn't that depend on the biases in question? Many argue that homogenous societies do many things better. And part of homogeneity is sharing same set of biases.

Re: Where the goblins came from

#290

Post by honzaik »

ahmadyan wrote: post #161
postalcoder wrote: post #17

Would love if OpenAI did more of these types of posts. Off the top of my head, I'd like to understand: - The sepia tint on images from gpt-image-1 - The obsession with the word "seam" as it pertains to coding Other LLM phraseology that I cannot unsee is Claude's "___ is the real unlock" (try google it or search twitter!). There's no way that this phrase is overrepresented in the training data, I don't remember people…

i just want to know where emdash came from, as it is quite rare to see it on the public internet, so it must have been synthetically added to the dataset.

although emdashes are not common on the internet, there are prevalent in books.
Post reply on HN