Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Telling GPT-4 you're scared or under pressure improves performance

aimodels.substack.com

Post reply on HN
Print view View original

61–70 of 255 posts

Re: Telling GPT-4 you're scared or under pressure improves performance

#61

Post by ignoramous »

drapado wrote: post #56
a_wild_dandan wrote: post #47

Earlier quoted context omitted.

The “statistical parrot” assertion is pretty thoroughly disproven by this point, but suppose we ignore the literature and just assume it’s true: what does it matter? “Real” people are time bombs too, for instance. Is there some predictive power that we gain by reducing LLM skills to mere token production side effects?

I'm curious in your statement, can you point to some papers where they addressed it?

(not op) The section A Path Forward in Managing AI Risks by Bengio et al cites a few papers: https://managing-ai-risks.com/

Re: Telling GPT-4 you're scared or under pressure improves performance

#62

Post by logicchains »

akoboldfrying wrote: post #52
toxik wrote: post #51

Earlier quoted context omitted.

It is not accurate to say that an LLM like ChatGPT predicts anything. It is trained to maximize a score function, so it is more like trying to win a game where the moves are word choices.

The game is predicting the next word a person would write.

Not after RLHF.

Re: Telling GPT-4 you're scared or under pressure improves performance

#63

Post by [deleted] »

a_wild_dandan wrote: post #47
intended wrote: post #19

I think this is the entry point needed to get peoples attention and explain: LLMs aren’t people, and emergent properties are being over extended. If LLMs are showing “better” performance when there are tokens that humans read as emotionally salient - Then the underlying text it’s trained on shows humans give better answers when emotionally salient context is provided. LLMs predict words. Any semantic validity is a si…

The “statistical parrot” assertion is pretty thoroughly disproven by this point, but suppose we ignore the literature and just assume it’s true: what does it matter? “Real” people are time bombs too, for instance. Is there some predictive power that we gain by reducing LLM skills to mere token production side effects?

[deleted]

Re: Telling GPT-4 you're scared or under pressure improves performance

#65

Post by kristiandupont »

mjburgess wrote: post #60
a_wild_dandan wrote: post #47

Earlier quoted context omitted.

The “statistical parrot” assertion is pretty thoroughly disproven by this point, but suppose we ignore the literature and just assume it’s true: what does it matter? “Real” people are time bombs too, for instance. Is there some predictive power that we gain by reducing LLM skills to mere token production side effects?

> The “statistical parrot” assertion is pretty thoroughly disproven by this point errr... all NNs are just optimisations of an associative probability objective: P(Y|X), they are by definition "statistical parrots". There isn't anything to prove or disprove. People offering prompts as evidence are people who fundamentally do not understand the basics. NNs aren't strange empirical objects, they're specified by mathema…

Well sure, but I think that same thing can be said about the human brain. Obviously at a whole different level of sophistication and all, but the two really do seem related, at least to me.

>People offering prompts as evidence are people who fundamentally do not understand the basics

This I agree with, but I also don't see anyone doing that in this thread?

Re: Telling GPT-4 you're scared or under pressure improves performance

#66

Post by williamcotton »

mjburgess wrote: post #60
a_wild_dandan wrote: post #47

Earlier quoted context omitted.

The “statistical parrot” assertion is pretty thoroughly disproven by this point, but suppose we ignore the literature and just assume it’s true: what does it matter? “Real” people are time bombs too, for instance. Is there some predictive power that we gain by reducing LLM skills to mere token production side effects?

> The “statistical parrot” assertion is pretty thoroughly disproven by this point errr... all NNs are just optimisations of an associative probability objective: P(Y|X), they are by definition "statistical parrots". There isn't anything to prove or disprove. People offering prompts as evidence are people who fundamentally do not understand the basics. NNs aren't strange empirical objects, they're specified by mathema…

You’re disregarding the emergent phenomena, which are not at all understood. There was a distinct and unpredicted jump in what can loosely be described as “cognitive abilities” between GPTs 2, 3, and 4, especially after some supervised techniques like RLHF.

Re: Telling GPT-4 you're scared or under pressure improves performance

#67

Post by intended »

tom-from-july wrote: post #27
intended wrote: post #19

I think this is the entry point needed to get peoples attention and explain: LLMs aren’t people, and emergent properties are being over extended. If LLMs are showing “better” performance when there are tokens that humans read as emotionally salient - Then the underlying text it’s trained on shows humans give better answers when emotionally salient context is provided. LLMs predict words. Any semantic validity is a si…

> That is why proof of concept LLM tools are mind blowing and production tools are semantic time bombs. This will be my new favorite quote for whenever someone tries to pitch his latest LLM idea

I have a whole list.

Syntactic validity is not semantic validity.

Word predictors not world state predictors

Text prediction not fact prediction

Frankly though the best answers are

1) let’s talk to infosec first

2) hey what’s the error rate ?

Re: Telling GPT-4 you're scared or under pressure improves performance

#68

Post by pk-protect-ai »

Havoc wrote: post #64

Is this an emotional trigger or does this simply steer it towards answer/content in the dataset where someone actually spent time answering because the poster made it clear it’s very important to them?

I believe the second option is correct. It steers towards the biases in the dataset in the same way that the uppercase words emphasize them. It all about probabilities, that is the common crawl for you ...

Re: Telling GPT-4 you're scared or under pressure improves performance

#69

Post by akoboldfrying »

logicchains wrote: post #62
akoboldfrying wrote: post #52

Earlier quoted context omitted.

The game is predicting the next word a person would write.

Not after RLHF.

I'd say RLHF bends the game towards predicting what words a "helpful", "respectful" person would write next, for values of "helpful" and "respectful" that vary according to each person involved in scoring (but which are carefully shaped by the people choosing those people, and paying for their time).

Re: Telling GPT-4 you're scared or under pressure improves performance

#70

Post by famouswaffles »

a_wild_dandan wrote: post #47
intended wrote: post #19

I think this is the entry point needed to get peoples attention and explain: LLMs aren’t people, and emergent properties are being over extended. If LLMs are showing “better” performance when there are tokens that humans read as emotionally salient - Then the underlying text it’s trained on shows humans give better answers when emotionally salient context is provided. LLMs predict words. Any semantic validity is a si…

The “statistical parrot” assertion is pretty thoroughly disproven by this point, but suppose we ignore the literature and just assume it’s true: what does it matter? “Real” people are time bombs too, for instance. Is there some predictive power that we gain by reducing LLM skills to mere token production side effects?

>Is there some predictive power that we gain by reducing LLM skills to mere token production side effects?

No. If anything, we lose predictive power which is why it's extra silly.

Post reply on HN