Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Where the goblins came from

openai.com

Post reply on HN
Print view View original

641–650 of 699 posts

Re: Where the goblins came from

#641

Post by lwansbrough »

fragmede wrote: post #633
lwansbrough wrote: post #600

Earlier quoted context omitted.

Ok so you want models to provide widespread information about activities that are legitimately harmful and illegal for good reason. And that’s the same as censoring a country’s violent history to you guys? How intellectually dishonest.

It means they have the same levers somewhere in the training process. Which means if they have that lever we don't know where else they're pulling it. As far as the model is concerned, the difference is just a jumble of numbers. Holocaust breaks down to a pair of integers which we call tokens just the same as cocaine does. We, as humans, ascribe different levels of meaning to those words, but as far as the model's co…

Do you have any actual examples of political history being actively censored by western models? Or are we just doing hypotheticals for fun?

Re: Where the goblins came from

#642

Post by lwansbrough »

gizajob wrote: post #267
AlfeG wrote: post #248

Earlier quoted context omitted.

And there is no "censor" in the USA models at all!

Of course there is. Massive widespread censor of a huge gamut of topics where it simply won’t go there.

Please point to an example where the information (or more importantly its practical application) is both censored but is also not legitimately harmful and/or illegal.

Re: Where the goblins came from

#643

Post by slater »

lwansbrough wrote: post #642
gizajob wrote: post #267

Earlier quoted context omitted.

Of course there is. Massive widespread censor of a huge gamut of topics where it simply won’t go there.

Please point to an example where the information (or more importantly its practical application) is both censored but is also not legitimately harmful and/or illegal.

[ "which opinions" goose meme :D ]

Re: Where the goblins came from

#644

Post by krsw »

sarchertech wrote: post #601
wincy wrote: post #591

Earlier quoted context omitted.

Smoking cigarettes didn’t really matter for as long as we were regularly burning wood for fuel. Turns out just burning pretty much anything and breathing in the particles is really bad for you. Makes sense we didn’t realize it was bad until we stopped burning logs and coal for home heating and cooking.

Cigarettes actually are uniquely bad when it comes to lung cancer. Lung cancer was very rare in 1900 and before when everyone was still burning wood or coal for warmth and cooking. Lung cancer rates didn’t take off until cigarette popularity exploded after WWI. Chewing tobacco also causes mouth cancer, so there’s more to it than just inhaling byproducts of combustion.

From what I remember reading chewing tobacco is orders of magnitude less cancer causing than smoking. So much so, that some groups see it as harmful to lump it in with smoking or vaping. If you really need some nic, popping a zyn is probably the least harmful way to get it.

Re: Where the goblins came from

#645

Post by lwansbrough »

tardedmeme wrote: post #254
lwansbrough wrote: post #216

Earlier quoted context omitted.

Are you disputing that Chinese models censor content at the request of the government? https://i.imgur.com/cVtLuj1.jpeg The absence of information is also Xi Jinping Thought.

All models censor content at the request of the government. Even the models you can download do it.

Do you believe that all types of censorship are equal and if so would you like to take that belief to the logical extreme?

Re: Where the goblins came from

#646

Post by djhn »

krsw wrote: post #644
sarchertech wrote: post #601

Earlier quoted context omitted.

Cigarettes actually are uniquely bad when it comes to lung cancer. Lung cancer was very rare in 1900 and before when everyone was still burning wood or coal for warmth and cooking. Lung cancer rates didn’t take off until cigarette popularity exploded after WWI. Chewing tobacco also causes mouth cancer, so there’s more to it than just inhaling byproducts of combustion.

From what I remember reading chewing tobacco is orders of magnitude less cancer causing than smoking. So much so, that some groups see it as harmful to lump it in with smoking or vaping. If you really need some nic, popping a zyn is probably the least harmful way to get it.

Do these… ”groups”, which would rather you not lump chewing tobacco together with other tobacco products… manufacture and sell chewing tobacco?

Re: Where the goblins came from

#647

Post by jagged-chisel »

dreis_sw wrote: post #480
jagged-chisel wrote: post #387

Earlier quoted context omitted.

An important stepping stone, perhaps. But I don’t think the final AGI thing will necessarily contain LLMs.

History shows continuous evolution, there won't be a "final AGI thing". The definition of AGI is so vague anyways that any conversation around it is hardly useful. 5 years ago, what we have today would have been considered AGI.

> 5 years ago, what we have today would have been considered AGI.

Were it you could pipe today’s LLM to an interface usable by someone 5 years ago, they might be impressed by the incremental improvement, but it would be obvious soon enough that it’s still not AGI.

Re: Where the goblins came from

#648

Post by mediaman »

dakolli wrote: post #357
mediaman wrote: post #78

Earlier quoted context omitted.

It has trained on vast amounts of content that contains the concept of self, of course the idea of self is emergent. And autoregressive LLMs are not stateless.

of course the idea of self is emergent You sound really sure of yourself, thousands of ML researchers would disagree with you that self awareness is emergent or at all apparent in large language models. You're literally psychotic if you think this is the case and you need to go touch grass.

I wrote that carefully; I'd recommend you re-read more slowly it instead of calling me "psychotic" and telling me that I need to "touch grass."

Of course they have the idea of self! The millions of pages of human text contains it; it is baked into the weights, just like the knowledge of the taste of Cheetos is baked in despite their lack of any taste buds. Their knowledge of it does not mean the neural net is actually a conscious creature or truly has a self.

I assume you do not call people psychotic to their face in real life, because it's mean. Please next time take a pause and consider if there is literally any other way you could communicate your potential disagreement.

Re: Where the goblins came from

#649

Post by krackers »

in-silico wrote: post #146
krackers wrote: post #44

Earlier quoted context omitted.

I wish the blog mentioned more about why exactly training for nerdy personality rewarded mention of goblins. Since it's probably not a deterministic verifiable reward, at their level the reward model itself is another LLM. But this just pushes the issue down one layer, why did _that_ model start rewarding mentions of goblin?

Either someone hard-coded it in a system prompt to the reward model (similar to how they hard-coded it out), or the reward model mixed up some kind of correlation/causation in the human preference data (goblins are often found in good responses != goblins make responses good). It's also possible that human data labellers really did think responses with goblins were better (in small doses).

>Someone hard-coded it in a system prompt to the reward model

I doubt this is the case, if so it wouldn't have taken an investigation to try to trace the root cause.

Re: Where the goblins came from

#650

Post by cyanydeez »

xerox13ster wrote: post #574
cyanydeez wrote: post #559

Earlier quoted context omitted.

Pack it in Anthropologists! No longer are you allowed to study pottery, knots, shelters or any of the other human-esque things! They're not human! What a bizarre understanding of what an anthropologist does.

Those are all things made by humans and therefore human constructs. The language and culture they are talking about studying would not be made by humans, they would be made by synthetics. I'm just saying, don't call the study of an extraterrestrial alien culture and its constructs and artifacts "anthropology", or even xenoanthropology (the extraterrestrial equivalent of AInthropology) --unless the extraterrestrials a…

llms are an artifact as pure as can he of human social langauge.
Post reply on HN