Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Telling GPT-4 you're scared or under pressure improves performance

aimodels.substack.com

Post reply on HN
Print view View original

171–180 of 255 posts

Re: Telling GPT-4 you're scared or under pressure improves performance

#171

Post by famouswaffles »

l5870uoo9y wrote: post #139
famouswaffles wrote: post #116

Earlier quoted context omitted.

LLMs are not lookup tables. They aren't "matching" in that way. There's this idea of "supercharged interpolation", the idea that they just take out certain texts and switch out words or whatever, that is not true. https://arxiv.org/abs/2110.09485

I was under the impression that LLMs vectorizes text and based on the user input tries to guess the next character based on a comparison of vectors?

That would be a Misconception.

Most Models are a two part process.

Training and Inference (when you sample the model after it is trained).

The text during training is tokenizer and embedded which basically just means it is broken down into a little more parts and vectorized.

At the start,the model is acomplete blank state. It has a bunch of neurons/parameters that do nothing.

How does training work ?

The model first gets some preceding text and tries to make a prediction of what might follow. It fails predictably. Now here's the rub. That failures helps. The model makes some changes to its parameters to reflect this failure. Now the model is just a tiny bit better. Rinse and repeat.

The weights/parameters/neurons are not that data re-encoded. They are more like instructions on how to make predictions based on what it has learnt so far.

Is it possible to memorize data ? Yes. Is that going to happen for the vast majority of data it is trained on ? Absolutely not.

Here's the interesting thing. After a certain scale of data, blatant memorization becomes a hindrance. It becomes harder to memorize every new thing it sees than to just learn how to make good predictions. So it doesn't memorize.

What I believe you were alluding to is how "attention" works in transformers.

When you pass some text into a transformer, basically the model makes a comparison of how each token relates to every other token in the text.

For example, let's say you give it the text.

"His child's name is John. He is probably a____"

How is "his" related to "child's" and "name" and "is" etc for each word and each combination. It uses this to aid predictions. How it uses this and the nuances of the comparisons are learnt in training.

Re: Telling GPT-4 you're scared or under pressure improves performance

#172

Post by besfriendt »

intended wrote: post #19

I think this is the entry point needed to get peoples attention and explain: LLMs aren’t people, and emergent properties are being over extended. If LLMs are showing “better” performance when there are tokens that humans read as emotionally salient - Then the underlying text it’s trained on shows humans give better answers when emotionally salient context is provided. LLMs predict words. Any semantic validity is a si…

>That is why proof of concept LLM tools are mind blowing and production tools are semantic time bombs.

What do you mean by this?

Re: Telling GPT-4 you're scared or under pressure improves performance

#173

Post by famouswaffles »

mjburgess wrote: post #170
famouswaffles wrote: post #166

Earlier quoted context omitted.

>Since it's incapable of actually summarising financial data. It's only capable of selecting combinations of pieces of its training set. Third completely off misconception from you today. This is not at all what it is doing. "Supercharged Interpolation" is false and makes no sense. It's not a lookup table either. It doesn't memorize enough of what it needs to to make your assertion possible. https://arxiv.org/abs/211…

at 500gb, you can store nearly everything ever written -- let alone compressed. all statistical learning is a variation on k-nn (see the relevant paper on this) but likewise this is obvious a priori k-nn is the ideal learner, and a good starting point for analysis the question for any given system is: what is the learning space, what is the distance function, and how many points are being considered NNs set up a comp…

>at 500gb, you can store nearly everything ever written -- let alone compressed.

No you cannot.

>That's just what they do -- there isn't any other mechanism here.

That's not what they do. They are many papers now showing ICL demonstrating some kind of optimization method during inference which would not be happening if all they did was retrieval.

I'm come to realize you don't know what you're talking about. Your level of denial is scary to see.

Re: Telling GPT-4 you're scared or under pressure improves performance

#174

Post by mjburgess »

famouswaffles wrote: post #173
mjburgess wrote: post #170

Earlier quoted context omitted.

at 500gb, you can store nearly everything ever written -- let alone compressed. all statistical learning is a variation on k-nn (see the relevant paper on this) but likewise this is obvious a priori k-nn is the ideal learner, and a good starting point for analysis the question for any given system is: what is the learning space, what is the distance function, and how many points are being considered NNs set up a comp…

>at 500gb, you can store nearly everything ever written -- let alone compressed. No you cannot. >That's just what they do -- there isn't any other mechanism here. That's not what they do. They are many papers now showing ICL demonstrating some kind of optimization method during inference which would not be happening if all they did was retrieval. I'm come to realize you don't know what you're talking about. Your leve…

just do the calculation yourself: how many books is 500gb at, say, a few bits per character?

more than all every written -- and so on

perhaps apply a single drop of scepticism to this credulity

even, just ask chatgpt to repeat the first paragraph of some book -- say, a dickens novel

Re: Telling GPT-4 you're scared or under pressure improves performance

#175

Post by famouswaffles »

mjburgess wrote: post #169
famouswaffles wrote: post #165

Earlier quoted context omitted.

>How does associative statical modelling do? It takes averages of past shadows, and calls the cause of the shadow that average: this is pseudoscience. Quite correct! Throw it all away. It does not work this way at all. In any sense. For one thing, it does not try to draw shadows. This would not be possible of so. https://www.pnas.org/doi/full/10.1073/pnas.2016239118 Transformers or predictors are not trying to draw s…

Unsupervised learning on discrete data is just ensembling modes. But let's look at how that helps in some cases. So if we already know the object is a cup, and we know how it's positioned, then its shadow is an actual guide to its particular geometry. So in cases where we have enough a priori scientific information, we can rig datasets (shadows) to be informative of the target domain. Here the target is discrete: say…

You are still getting this wrong. You don't need to "rig" anything. I've linked a paper. Read it.

They just fed protein sequences. They did not alter the architecture in any way. To the transformer, it may as well have been any random assemblage of letters and numbers.

Functions like secondary structure, contacts, and biological activity were found because those things are implicit in the creation of the data, not because the model was "rigged" in any way.

Re: Telling GPT-4 you're scared or under pressure improves performance

#176

Post by Lord-Jobo »

pbhjpbhj wrote: post #95
RheingoldRiver wrote: post #17

Earlier quoted context omitted.

> I have to push back really hard against this stuff Based on ChatGPT's answers, OpenAI think you should be saying thanks to it, because it helps you have a more natural conversation, which encourages you as the user to send more natural/productive prompts when asking real questions. As for asking it dumb questions, how often do you use Google as a spellcheck? I wouldn't consider this any different.

>how often do you use Google as a spellcheck? // When Word tells me a word is wrong, but I'm almost certain it's right, and it turns out not to be in the Microsoft dictionary somehow. I mean, I have an extensive vocabulary, but not more extensive than a decent British-English dictionary.

I've had both Word and Google services fuck up pretty common words or spellings. Just completely unaware of them. Google specifically will miss a ton of words when using Gboard, but if you go to Docs it will know the word. How in the world are they not using the same dictionaries.

Re: Telling GPT-4 you're scared or under pressure improves performance

#177

Post by omarfarooq »

mjburgess wrote: post #149
ethbr1 wrote: post #122

Earlier quoted context omitted.

"Are not" is the rub here. They 100% are not those things... but they also approximate them well-enough to be functionally useful. I.e. the high-dimensional curve-fitting / compression conceptualization of ML, which intuitively expresses both its strengths and weaknesses. If "it" is represented in the data set (explicitly or implicitly), the "curve" will fit to that property. Simultaneously, the "curve" is approximat…

They're statistical approximations of these things -- that's really the rub. You can approximate a human capacity, say theory-of-mind, with another kind of ape: play some hide-and-seek game. You can approximate the knowledge of a trivia-master with a child and a trivia book. These are quite different sorts of approximations. A 1/100th scale bridge build to stand for a real one is quite different than taking some prio…

ChatGPT explains what this text means to those of us may not readily grasp it: https://chat.openai.com/share/ce79b704-705c-4add-92af-584c20...

Re: Telling GPT-4 you're scared or under pressure improves performance

#178

Post by omarfarooq »

mjburgess wrote: post #160
ethbr1 wrote: post #157

Earlier quoted context omitted.

We're in agreement on the nature of SOTA and the world, I think. I'd only add that certainty requirements for real-world target applications can differ substantially. I.e. engineering vs art. A toy box that gives magic answers 85% of the time is incredibly useful in some scenarios -- e.g. seeding the beginning of a manual research process with initial topics. Which seems a general rule of thumb for deploying GenAI in…

That's exactly my advise also. Take a risk profile over the predictions, and use them only when the risk of error is low. You can't predict almost anything anyway. The relevant distribution for acting is a utility+risk distribution over various sets of predictions. If you compute that, most of ML/AI isnt very useful for most of anyone. It's kinda interesting that generative AI finally achieved something here, given o…

ChatGPT responds: https://chat.openai.com/c/7c4ae31a-3391-4496-8db3-a92a58a4e1...

Re: Telling GPT-4 you're scared or under pressure improves performance

#179

Post by mjburgess »

famouswaffles wrote: post #175
mjburgess wrote: post #169

Earlier quoted context omitted.

Unsupervised learning on discrete data is just ensembling modes. But let's look at how that helps in some cases. So if we already know the object is a cup, and we know how it's positioned, then its shadow is an actual guide to its particular geometry. So in cases where we have enough a priori scientific information, we can rig datasets (shadows) to be informative of the target domain. Here the target is discrete: say…

You are still getting this wrong. You don't need to "rig" anything. I've linked a paper. Read it. They just fed protein sequences. They did not alter the architecture in any way. To the transformer, it may as well have been any random assemblage of letters and numbers. Functions like secondary structure, contacts, and biological activity were found because those things are implicit in the creation of the data, not be…

the rigging occurs in the design of the data generation process, ie., those experiments which lead to these datasets

that is where the science occurs -- the data analysis is just an administrative task after science has taken place

Re: Telling GPT-4 you're scared or under pressure improves performance

#180

Post by intended »

ethbr1 wrote: post #154
intended wrote: post #148

Earlier quoted context omitted.

In essence: “You cant know in advance how far the model can approximate semantic patterns” So claiming that out-of-sample performance is a mirage, would be a bridge too far?

Maybe "a mirage that might actually be true"? Which is a terrible thing to rely on! Unless it's usually true?

That measurement is the core of my current tasks. If you don’t know the error rate - then what are you doing ?
Post reply on HN