Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Where the goblins came from

openai.com

Post reply on HN
Print view View original

101–110 of 699 posts

Re: Where the goblins came from

#101

Post by TychoCelchuuu »

albert_e wrote: post #37

If a tiny misconfiguration of reward system can cause such noticeable annoyance ... What dangers lurk beneath the surface. This is not funny.

This is a worry that people have been talking about in various forms for a while now, and I think it's a gigantic one. The only reason this was caught is that the quirk was a very noticeable verbal one. When words like "goblin" and "gremlin" pop up it is easy for us to spot. If the quirk takes another shape (say, ranking certain people with certain features as less trustworthy) it might be too subtle or too weird for us to notice it. Would I ever notice if ChatGPT consistently rates people born in June to be untrustworthy?

Here is an academic paper discussing this kind of worry: https://link.springer.com/article/10.1007/s11023-022-09605-x

Re: Where the goblins came from

#102

Post by tdeck »

krackers wrote: post #22
postalcoder wrote: post #17

Would love if OpenAI did more of these types of posts. Off the top of my head, I'd like to understand: - The sepia tint on images from gpt-image-1 - The obsession with the word "seam" as it pertains to coding Other LLM phraseology that I cannot unsee is Claude's "___ is the real unlock" (try google it or search twitter!). There's no way that this phrase is overrepresented in the training data, I don't remember people…

>with the word "seam" as it pertains to coding I thought this was an established term when it comes to working with codebases comprised of multiple interacting parts. https://softwareengineering.stackexchange.com/questions/1325...

I can't say it isn't, but I have been writing code since about 2004 and this is the first time I've become aware that this is a thing.

Re: Where the goblins came from

#103

Post by dexwiz »

tdeck wrote: post #96
christoph wrote: post #82

Earlier quoted context omitted.

Does nobody else laugh that a company supposedly worth more than almost anything else at the moment, is basically hacking around a load of text files telling their trillion dollar wonder machine it absolutely must stop talking to customers about goblins, gremlins and ogres? The number one discussion point, on the number one tech discussion site. This literally is, today, the state of the art. McKenna looks more corre…

Is this the "prompt engineering" that I keep hearing will be an indispensable job skill for software engineers in the AI-driven future? I had better start learning or I'll be replaced by someone who has.

Prompt engineering is mostly structured thought. Can you write a lab report? Can you describe the who, what, when, where, and why of a problem and its solution?

You can get it to work with one off commands or specific instructions, but I think that will be seen as hacks, red flags, prompt smells in the long term.

Re: Where the goblins came from

#104

Post by helloplanets »

jumploops wrote: post #20

TIL gremlins weren’t just used to explain mysterious mechanical failures in airplanes, it’s the origin story of the term ‘gremlin’ itself[0]. I had always assumed there was some previous use of the term, neat! [0] https://en.wikipedia.org/wiki/Gremlin

So the word is actually semantically very close to "bug"! I guess we could still be using it, but the word's just too long for something that is one of the most used terms in software development.

At this point, picking that specific word is not at all a random quirk, as it's using the word literally like it's originally intended to be used.

Re: Where the goblins came from

#105

Post by tdeck »

dexwiz wrote: post #103
tdeck wrote: post #96

Earlier quoted context omitted.

Is this the "prompt engineering" that I keep hearing will be an indispensable job skill for software engineers in the AI-driven future? I had better start learning or I'll be replaced by someone who has.

Prompt engineering is mostly structured thought. Can you write a lab report? Can you describe the who, what, when, where, and why of a problem and its solution? You can get it to work with one off commands or specific instructions, but I think that will be seen as hacks, red flags, prompt smells in the long term.

If I could do those things, I wouldn't be using an LLM to write for me, now would I?

Re: Where the goblins came from

#107

Post by card_zero »

ggsp wrote: post #93
xerox13ster wrote: post #23

Earlier quoted context omitted.

Anthro means human and these are not human. Please do not use anthropology or any derivative of the word to refer to non-human constructs. I suggest Synthetipologists, those who study beings of synthetic origin or type, aka synthetipodes, just as anthropologists study Anthropodes

Agree with your sentiment, I think synthetologist (σύνθετος/synthetos + λογία/logia) flows better. The plural of anthropos is anthropoi, not anthropodes.

But since when is there a synthetos? Since right now, I guess. Shrug But you know it's from the same root as thesis, and synthesis (or a more proper ancient Greek spelling) is the noun and doesn't end in -os.

σύνθεσις (súnthesis, “a putting together; composition”), says Wiktionary.

Oh wait there is a σύνθετος, but it's an adjective for "composite". Hmm, OK. Modern Greek, looks like.

Re: Where the goblins came from

#108

Post by DespairYeMighty »

pdntspa wrote: post #87
postalcoder wrote: post #17

Would love if OpenAI did more of these types of posts. Off the top of my head, I'd like to understand: - The sepia tint on images from gpt-image-1 - The obsession with the word "seam" as it pertains to coding Other LLM phraseology that I cannot unsee is Claude's "___ is the real unlock" (try google it or search twitter!). There's no way that this phrase is overrepresented in the training data, I don't remember people…

The number of things that Claude has told me are 'load-bearing' or 'belt-and-suspenders' is... very load-bearing

for me, doing the heavy lifting is doing the heavy lifting

Re: Where the goblins came from

#109

Post by vidarh »

postalcoder wrote: post #17

Would love if OpenAI did more of these types of posts. Off the top of my head, I'd like to understand: - The sepia tint on images from gpt-image-1 - The obsession with the word "seam" as it pertains to coding Other LLM phraseology that I cannot unsee is Claude's "___ is the real unlock" (try google it or search twitter!). There's no way that this phrase is overrepresented in the training data, I don't remember people…

Claude, at least 4.5, not checked recently, has/had an obsession with the number 47 (or numbers containing 47). Ask it to pick a random time or number, or write prose containing numbers, and the bias was crazy.

Also "something shifted" or "cracked".

Re: Where the goblins came from

#110

Post by oofbey »

Wherein OpenAI admits they have very little understanding of how their models’ personality develops. And implicitly admit it’s not all that important to them, except when it gets so out of hand that they get caught making blunt corrections.
Post reply on HN