If a tiny misconfiguration of reward system can cause such noticeable annoyance ... What dangers lurk beneath the surface. This is not funny.
For every gremlin spotted, many remain unseen...
Where the goblins came from
51–60 of 699 posts
Re: Where the goblins came from
#52Earlier quoted context omitted.
> Absolutely terrifying that OpenAI is just tossing around that such subtle training biases were hard enough to contain it had to be added to system prompt. May I introduce you to homo sapiens , a species so vulnerable to such subtle (or otherwise) biases (and affiliations) that they had to develop elaborate and documented justice systems to contain the fallouts? :)
We’re really not that vulnerable to such things as a species, because we as individuals all have our own minds and our own sets of biases that cancel out and get lost in the noise. If we all had the exact same bias then it would be a huge problem.
> If we all had the exact same bias then it would be a huge problem.
And may I introduce you to "groupthink" :))
Re: Where the goblins came from
#53Earlier quoted context omitted.
It was always funny how easy it was to spot the people using a Studio Ghibli style generated avatar for their Discord or Slack profile, just from that yellow tinging. A simple LUT or tone-mapping adjustment in Krita/Photoshop/etc. would have dramatically reduced it. The worst was you could tell when someone had kept feeding the same image back into chatgpt to make incremental edits in a loop. The yellow filter would…
For context, an example of what happens when you feed the same image back in repeatedly: https://www.instagram.com/reels/DJFG6EDhIHs/
Haha fantastic. I'd love to see a comparison reel of that same image-loop for the entire image gen series (gpt-image-1, gpt-image-1.5, gpt-image-2).
Re: Where the goblins came from
#54Earlier quoted context omitted.
Anthro means human and these are not human. Please do not use anthropology or any derivative of the word to refer to non-human constructs. I suggest Synthetipologists, those who study beings of synthetic origin or type, aka synthetipodes, just as anthropologists study Anthropodes
> Please do not use anthropology or any derivative of the word to refer to non-human constructs So you, for one, do not welcome our new robot overlords? A rather risky position to adopt in public, innit ;-)
I’ve already had my Roko’s basilisk existential breakdown a decade ago, so I don’t really care one way or the other.
I just wanna point out that I only called them non-human and I am asking for a precision of language.
Re: Where the goblins came from
#55Earlier quoted context omitted.
Anthro means human and these are not human. Please do not use anthropology or any derivative of the word to refer to non-human constructs. I suggest Synthetipologists, those who study beings of synthetic origin or type, aka synthetipodes, just as anthropologists study Anthropodes
> Synthetipologists, those who study Synthetic beings. I see you took the prudent approach of recognizing the being-ness of our future overlords :) ("being" wasn't in your first edit to which I responded below...) Still, a bit uninspired, methinks. I like AInthropologist better, and my phone's keyboard appears to have immediately adopted that term for the suggestions line. Who am I to fight my phone's auto-suggest :-…
They are state machines so they have a state of being therefore they are beings. Living is an entirely different argument.
Re: Where the goblins came from
#56Re: Where the goblins came from
#57> the evidence suggests that the broader behavior emerged through transfer from Nerdy personality training. > The rewards were applied only in the Nerdy condition, but reinforcement learning does not guarantee that learned behaviors stay neatly scoped to the condition that produced them > Once a style tic is rewarded, later training can spread or reinforce it elsewhere, especially if those outputs are reused in super…
Anthro means human and these are not human. Please do not use anthropology or any derivative of the word to refer to non-human constructs. I suggest Synthetipologists, those who study beings of synthetic origin or type, aka synthetipodes, just as anthropologists study Anthropodes
It is not in any sense of the word a being, it's a sophisticated generator that relies entirely on what you feed it.
Re: Where the goblins came from
#58This is funny because it’s a silly topic, but I think it shows something extremely seriously wrong with llms. The goblins stand out because it’s obvious. Think of all the other crazy biases latent in every interaction that we don’t notice because it’s not as obvious. Absolutely terrifying that OpenAI is just tossing around that such subtle training biases were hard enough to contain it had to be added to system promp…
I think it's extraordinarily telling that people are capable of being reflexively pessimistic in response to the goblin plague. It's like something Zitron would do.
This story is wonderful.
Re: Where the goblins came from
#59> the evidence suggests that the broader behavior emerged through transfer from Nerdy personality training. > The rewards were applied only in the Nerdy condition, but reinforcement learning does not guarantee that learned behaviors stay neatly scoped to the condition that produced them > Once a style tic is rewarded, later training can spread or reinforce it elsewhere, especially if those outputs are reused in super…
I call myself an AI theologian. I don't think humans are smart enough to be AInthropologists. The models are too big for that. Nobody really understands what's truly going on in these weights, we can only make subjective interpretations, invent explanations, and derive terminal scriptures and morals that would be good to live by. And maybe tweak what we do a little bit, like OpenAI did here.
I don’t see much of a distinction from anthropology
Re: Where the goblins came from
#60Earlier quoted context omitted.
Seams, spirals, codexes, recursion, glyphs, resonance, the list goes on and on.
Ask any LLM for 10 random words and most of them will give you the same weird words every time.
If you lower the temperature setting, it really will be the same 10 words every single attempt. :p