Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Where the goblins came from

openai.com

Post reply on HN
Print view View original

141–150 of 699 posts

Re: Where the goblins came from

#141

Post by eptcyka »

tdeck wrote: post #105
dexwiz wrote: post #103

Earlier quoted context omitted.

Prompt engineering is mostly structured thought. Can you write a lab report? Can you describe the who, what, when, where, and why of a problem and its solution? You can get it to work with one off commands or specific instructions, but I think that will be seen as hacks, red flags, prompt smells in the long term.

If I could do those things, I wouldn't be using an LLM to write for me, now would I?

You don’t let the LLM write prise for you, you get it to translate natural language into code somewhat coherently.

Re: Where the goblins came from

#142

Post by tdeck »

eptcyka wrote: post #141
tdeck wrote: post #105

Earlier quoted context omitted.

If I could do those things, I wouldn't be using an LLM to write for me, now would I?

You don’t let the LLM write prise for you, you get it to translate natural language into code somewhat coherently.

In this instance I'm assuming most of the "goblin" references were in prose rather than in source code, so the goal of this particular prompt edit was directed toward making the prose better.

Re: Where the goblins came from

#143

Post by tonypapousek »

nomilk wrote: post #4

> We unknowingly gave particularly high rewards for metaphors with creatures. I recall a math instructor who would occasionally refer to variables (usually represented by intimidating greek letters) as "this guy". Weirdly, the casual anthropomorphism made the math seem more approachable. Perhaps 'metaphors with creatures' has a similar effect i.e. makes a problem seem more cute/approachable. On another note, buzzword…

I had a calc prof years ago that would say f of cow, or f of pig instead of x or g. It was more engaging trying to keep track of f of pig of cow than the single-letter func names.

He was one of those classic types; you could always catch him for a quick chat 4 minutes before class, as he lit up a cig by the front door. Back when they allowed smoking on campus, anyway.

Re: Where the goblins came from

#144

Post by BOOSTERHIDROGEN »

bahadiraydin wrote: post #138

I'd like to see them explain why AI have so distinctive writing style that is very easy to detect most of the time. Even though, it had immense progress in coding, it didn't get better at writing.

The vector syncopancy is very unformal for human writing which programming itself already a "formal" language.

Re: Where the goblins came from

#145

Post by starshadowx2 »

thatguymike wrote: post #114
nomilk wrote: post #4

> We unknowingly gave particularly high rewards for metaphors with creatures. I recall a math instructor who would occasionally refer to variables (usually represented by intimidating greek letters) as "this guy". Weirdly, the casual anthropomorphism made the math seem more approachable. Perhaps 'metaphors with creatures' has a similar effect i.e. makes a problem seem more cute/approachable. On another note, buzzword…

A decade ago I gave a presentation on automata theory. I demonstrated writing arbitrary symbols to tape with greek letters, just like I’d learned at university. The audience was pretty confused and didn’t really grok the presentation. A genius communicator in the audience advised me to replace the greek letters with emoji… I gave the same presentation to the same demographic audience a week later and it was a smash h…

This is sortof like how Only Connect switched from using Greek letters to Egyptian hieroglyphs. I'm not sure if it was a joke or not but it was said that viewers complained that the Greek letters were "too pretentious" and obviously the hieroglyphs weren't.

Re: Where the goblins came from

#146

Post by in-silico »

krackers wrote: post #44
JoshTriplett wrote: post #5

A plausible theory I've seen going around: https://x.com/QiaochuYuan/status/2049307867359162460

I wish the blog mentioned more about why exactly training for nerdy personality rewarded mention of goblins. Since it's probably not a deterministic verifiable reward, at their level the reward model itself is another LLM. But this just pushes the issue down one layer, why did _that_ model start rewarding mentions of goblin?

Either someone hard-coded it in a system prompt to the reward model (similar to how they hard-coded it out), or the reward model mixed up some kind of correlation/causation in the human preference data (goblins are often found in good responses != goblins make responses good). It's also possible that human data labellers really did think responses with goblins were better (in small doses).

Re: Where the goblins came from

#147

Post by eterm »

postalcoder wrote: post #17

Would love if OpenAI did more of these types of posts. Off the top of my head, I'd like to understand: - The sepia tint on images from gpt-image-1 - The obsession with the word "seam" as it pertains to coding Other LLM phraseology that I cannot unsee is Claude's "___ is the real unlock" (try google it or search twitter!). There's no way that this phrase is overrepresented in the training data, I don't remember people…

"is the real" is such a strong Claude tell, whenever I encounter it, it makes me question what i'm reading.

Another I've noticed more recently is a slight obsession over refering to "Framing".

Re: Where the goblins came from

#150

Post by eterm »

postalcoder wrote: post #35
krackers wrote: post #22

Earlier quoted context omitted.

>with the word "seam" as it pertains to coding I thought this was an established term when it comes to working with codebases comprised of multiple interacting parts. https://softwareengineering.stackexchange.com/questions/1325...

thanks for this. > the term originates from Michael Feathers Working Effectively with Legacy Code I haven’t read the book but, taking the title and Amazon reviews at face value, I feel like this embodies Codex’s coding style as a whole. It treats all code like legacy code.

It's been a long time since I read it, but it was one of the better books I've read. It changed my approach to how to think about old code-bases.
Post reply on HN