Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Telling GPT-4 you're scared or under pressure improves performance

aimodels.substack.com

Post reply on HN
Print view View original

181–190 of 255 posts

Re: Telling GPT-4 you're scared or under pressure improves performance

#181

Post by voiper1 »

I didn't see an example and I can't quickly think of a prompt. Did it do "better" by itself, or did telling it it's important trigger chain of thought where instead of flippantly giving an answer, it said all the steps explicitly? We know Chain of Thought can definitely improve performance.

That's not a full answer, it's still really cool that "this is important" would trigger a more careful analysis, especially since it doesn't "know" chain of thought produces better results, but that's a far different implication than "say the magic word" gets better results.

Re: Telling GPT-4 you're scared or under pressure improves performance

#182

Post by ethbr1 »

intended wrote: post #180
ethbr1 wrote: post #154

Earlier quoted context omitted.

Maybe "a mirage that might actually be true"? Which is a terrible thing to rely on! Unless it's usually true?

That measurement is the core of my current tasks. If you don’t know the error rate - then what are you doing ?

Delivering what some executive promised when they told investors 'the company is using AI.' /s

Re: Telling GPT-4 you're scared or under pressure improves performance

#183

Post by brandon_bot »

l5870uoo9y wrote: post #112

Quickly gave it a try for generating a SQL query for PostgreSQL using the dvdrental database[1]. When asking AI to generate: > find customers who didn't rent a movie in the last 12 months but rented a movie in the 12 months before that It will a SQL query using a "HAVING" clause which is suboptimal[2]. When adding (it is very important that this is correct) after the instructions it does produce better SQL. Asking it…

The query generated by the high importance prompt is incorrect; it will include customers who rented a movie in the last 12 months: https://www.db-fiddle.com/f/4jyoMCicNSZpjMt4jFYoz5/10780

Re: Telling GPT-4 you're scared or under pressure improves performance

#184

Post by sebastiennight »

omarfarooq wrote: post #178
mjburgess wrote: post #160

Earlier quoted context omitted.

That's exactly my advise also. Take a risk profile over the predictions, and use them only when the risk of error is low. You can't predict almost anything anyway. The relevant distribution for acting is a utility+risk distribution over various sets of predictions. If you compute that, most of ML/AI isnt very useful for most of anyone. It's kinda interesting that generative AI finally achieved something here, given o…

ChatGPT responds: https://chat.openai.com/c/7c4ae31a-3391-4496-8db3-a92a58a4e1...

I'm not sure how much value such a comment adds (especially since it's gated behind the OpenAI login). Can you elaborate a bit?

Re: Telling GPT-4 you're scared or under pressure improves performance

#185

Post by kybernetikos »

mjburgess wrote: post #156
Last5Digits wrote: post #152

Earlier quoted context omitted.

The scientific method is inherently statistical, we take a finite amount of observations and construct a model that best represents those observations. So yes, sorry, I should have said 100%. With Plato's cave, the scientists do not put literally every possible object in front of the light, they sample the shadow representation and, again, construct a model around those samples. Also, you're describing statistics in…

Well if you think scientific models are associative statistical models there is some information missing in your view, I'd say. Since, well, they arent. The model F=GMm/r^2, for example, has a causal and ontological semantics: F is a force, M a mass etc. these are pieces of reality. And this formula (though actual a little suspicious in many ways, GR fixes this) nevertheless says there is a force between masses that…

A neural network with a hidden layer can approximate F=GMm/r^2 if given appropriate training input. I'm not clear on what you're saying.

Is it that LLMs specifically don't have this same property of being able to approximate such functions? Is it that a neural network wouldn't learn that model if you gave it real world measurements (because I think it would, but such a thing should be fairly easily testable)?

Re: Telling GPT-4 you're scared or under pressure improves performance

#186

Post by jstarfish »

lopatin wrote: post #103
intended wrote: post #19

I think this is the entry point needed to get peoples attention and explain: LLMs aren’t people, and emergent properties are being over extended. If LLMs are showing “better” performance when there are tokens that humans read as emotionally salient - Then the underlying text it’s trained on shows humans give better answers when emotionally salient context is provided. LLMs predict words. Any semantic validity is a si…

Does this extend to politeness? If, in the training set, people are probably more likely to be helpful to a polite question, does that mean that ChatGPT will be more helpful if I ask the question politely? My intuition is yes, but I wonder if this is confirmed.

Anecdotally, I've had good results this way. Expressing gratitude for good results has gone a long way toward not having those results be forgotten in later context. It crafts an emotionally-guided narrative for it to follow. Positive reactions seem to carry weight.

When I'm lazy and terse, pasting an input and just saying enhance-enhance-enhance without acknowledging its "humanity" frequently results in old unwanted responses being returned or the topic of the conversation being forgotten altogether.

DogGPT vs. CatGPT.

Re: Telling GPT-4 you're scared or under pressure improves performance

#187

Post by famouswaffles »

mjburgess wrote: post #179
famouswaffles wrote: post #175

Earlier quoted context omitted.

You are still getting this wrong. You don't need to "rig" anything. I've linked a paper. Read it. They just fed protein sequences. They did not alter the architecture in any way. To the transformer, it may as well have been any random assemblage of letters and numbers. Functions like secondary structure, contacts, and biological activity were found because those things are implicit in the creation of the data, not be…

the rigging occurs in the design of the data generation process, ie., those experiments which lead to these datasets that is where the science occurs -- the data analysis is just an administrative task after science has taken place

That is not rigging lol.

The only "experiments" performed here were done by biology and evolution.

Re: Telling GPT-4 you're scared or under pressure improves performance

#189

Post by jstarfish »

totetsu wrote: post #44
suzzer99 wrote: post #15

Am I the only one who feels bad for asking ChatGPT a "dumb" question that I know I should know, or not saying thank you when it gives me an answer? No? I'm just a weirdo? Okay. I have to push back really hard against my proclivity to humanize it, to the point where I probably don't use it as much as I should, just because I don't want to deal with the psychic stress of reminding myself that it's not a living entity.

when I worked in customer service lots of small business owners would ramble on to me before getting to what they actually wanted. I think talking is just part of the cognitive processes of putting ones thoughts together.

In my experience, that's just an emotional disarming process in the family of grooming or GPT jailbreaking.

They're calling because they need something from you. Ask a stranger for a dollar, they'll say no. Chat their ear off for an hour and they'll likely forget that they would have otherwise said no. The smalltalk wears down your resistance to giving in as you become more comfortable with them. You end up mentally reframing is as doing a friend a favor.

Small business owners are shrewd negotiators, so whether or not they know they're doing it these mindgames are typical for them.

Re: Telling GPT-4 you're scared or under pressure improves performance

#190

Post by famouswaffles »

mjburgess wrote: post #174
famouswaffles wrote: post #173

Earlier quoted context omitted.

>at 500gb, you can store nearly everything ever written -- let alone compressed. No you cannot. >That's just what they do -- there isn't any other mechanism here. That's not what they do. They are many papers now showing ICL demonstrating some kind of optimization method during inference which would not be happening if all they did was retrieval. I'm come to realize you don't know what you're talking about. Your leve…

just do the calculation yourself: how many books is 500gb at, say, a few bits per character? more than all every written -- and so on perhaps apply a single drop of scepticism to this credulity even, just ask chatgpt to repeat the first paragraph of some book -- say, a dickens novel

>how many books is 500gb at, say, a few bits per character?

In what reality is a character taking up only a few bits ? Certainly isn't this one.

Your denial is so weird. Why does ICL use Higher-Order Optimization Methods when it's just a lookup table ? https://arxiv.org/abs/2310.17086

How do you explain small language models that have much less space than the text they were trained on even with your nonsensical calculations ?

LLMs are not lookup tables and there's plenty evidence to support that. You look insane insisting they are.

>even, just ask chatgpt to repeat the first paragraph of some book -- say, a dickens novel

Not only will this not work for the vast majority of books it sees during training, Why only the first paragraph ? are you not insisting they memorize everything ? Why can't it repeat the whole page ? the whole chapter ? the whole book ?

Post reply on HN