Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4o's Memory Breakthrough – Needle in a Needlestack

nian.llmonpy.ai

Post reply on HN
Print view View original

91–100 of 256 posts

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#91

Post by djsjajah »

cma wrote: post #69
sftombu wrote: post #55

Earlier quoted context omitted.

I tested the LLMs to make sure they could not answer the questions unless the limerick was given to them. Other than 4o, they do very badly on this benchmark, so I don't think the test is invalidated by their training.

Why wouldn't it still be invalidated by it if it was indeed trained on it? The others may do worse and may or may not have been trained on it, but them failing on ititself doesn't imply 4o can do this well without the task being present in the corpus.

A better test would be to see if it can still answer the question if you just exclude the limerick for that answer. Having a bunch of limericks in the context window will make it "think" about all of the limericks it "knows".

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#92

Post by youssefabdelm »

jddj wrote: post #35
jddj wrote: post #30

Earlier quoted context omitted.

An elaborate Agatha Christie style whodunit, with a series of plot-twists and alibis which can be chopped off the end of the piece to modify who is the most likely suspect

Or a spot the difference. Generate 1000 generic facts about Alice and the same 1000 facts about Eve. Randomise the order and change one minor detail then ask how they differ.

That seems to go back in the direction of needle in the haystack again

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#93

Post by kaibee »

int_19h wrote: post #87
robbiep wrote: post #78

Earlier quoted context omitted.

I’m not involved in the space, but it seems to me that having a model, in particular a massive model, exposed to a corpus of text like a book in the training data would have very minimal impact. I’m aware that people have been able to return data ‘out of the shadows’ pf the training data but to my mind a model being mildly influenced by the weights between different words in this text hardly constitute hard recall, i…

It depends on how many times it had seen that text during training. For example, GPT-4 can reproduce ayats from the Quran word for word in both Arabic and English. It can also reproduce the Navy SEAL copypasta complete with all the typos.

Poe's "The Raven" also.

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#94

Post by kaibee »

19h wrote: post #26
Fernicia wrote: post #19

Earlier quoted context omitted.

But this content is presumably in its training set, no? I'd be interested if you did the same task for a collection of books published more recently than the model's last release.

To test this hypothesis, I just took the complete book "Advances in Green and Sustainable Nanomaterials" [0] and pasted it into the prompt, asking Gemini: "What absorbs thermal radiations and converts it into electrical signals?". It replied: "The text indicates that graphene sheets present high optical transparency and are able to absorb thermal radiations with high efficacy. They can then convert these radiations i…

Honestly I think testing these on fiction books would be more impressive. The graphene thing I'm sure shows up in some research papers.

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#95

Post by causal »

Aeolun wrote: post #77
sftombu wrote: post #72

Earlier quoted context omitted.

It can't answer the questions without the limericks in the prompt. The benchmark is to establish how well it uses the context window. For example, I just asked it "What is sought by the English top brass?". The answer from the limerick is "Cranberry glass" and 4o answers correctly when given the associated limerick once out of 2500+ limericks. However, without the limerick, 4o responded with: "The term "English top b…

Maybe if you tell it to pull the answer from a limerick instead of generally asking? Edit: Ok no, I tried giving it a whole bunch of hints, and it was just making stuff up that was completely unrelated. Even directly pointing it at the original dataset didn’t help.

Yeah I also tried to get it to complete some limericks from the dataset. Curiously it believed it had heard of the limerick but would then recite a hallucination.

So the good news is that the NIAN score might be real, bad news is you can't rely on it to know what it knows.

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#96

Post by causal »

sftombu wrote: post #60
cararemixed wrote: post #31

What's the chance that these limericks are now in the training set? As others mention, it'd be interesting to come up with a way to synthesize something sufficiently interesting so it always evades training fit.

Previous answer to this question: https://news.ycombinator.com/item?id=40361419

Your test is a good one but the point still stands that a novel dataset is the next step to being sure.

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#97

Post by Salgat »

robbiep wrote: post #78
nsagent wrote: post #54

Earlier quoted context omitted.

A number of people in my lab do research into long context evaluation of LLMs for works of fiction. The likelihood is very high that Moby Dick is in the training data. Instead the people in my lab have explored recently published books to avoid these issues. See BooookScore ( https://openreview.net/forum?id=7Ttk3RzDeu ) which was just presented at ICLR last week and FABLES ( https://arxiv.org/abs/2404.01261 ) a recen…

I’m not involved in the space, but it seems to me that having a model, in particular a massive model, exposed to a corpus of text like a book in the training data would have very minimal impact. I’m aware that people have been able to return data ‘out of the shadows’ pf the training data but to my mind a model being mildly influenced by the weights between different words in this text hardly constitute hard recall, i…

Remember, it's also trained on countless internet discussions and papers on the book.

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#98

Post by sftombu »

thorum wrote: post #43

The needle in the haystack test gives a very limited view of the model’s actual long context capabilities. It’s mostly used because early models were terrible at it and it’s easy to test. In fact, most recent models now do pretty good at this one task, but in practice, their ability to do anything complex drops off hugely after 32K tokens. RULER is a much better test: https://github.com/hsiehjackson/RULER > Despite a…

The models benchmarked by RULER do worse in needle in a needlestack. It will be interested to see how 4o does with RULER.

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#99

Post by bckr »

bearjaws wrote: post #81

I just used it to compare two smaller legal documents and it completely hallucinated that items were present in one and not the other. It did this on three discrete sections of the agreements. Using ctrl-f I was able to see that they were identical in one another. Obviously this is a single sample but saying 90% seems unlikely. They were around ~80k tokens total.

Yeah I asked for an estimate of the percentage of the US population that lives in the DMV area (DC, Maryland, Virginia) and it was off by 50% of the actual answer, which I only realized when I realized I shouldn’t trust its estimate for anything important

Re: GPT-4o's Memory Breakthrough – Needle in a Needlestack

#100

Post by mianos »

parrt wrote: post #2

The article shows how much better GPT-4o is at paying attention across its input window compared to GPT-4 Turbo and Claude-3 Sonnet. We've needed an upgrade to needle in a haystack for a while and this "Needle In A Needlestack" is a good next step! NIAN creates a prompt that includes thousands of limericks and the prompt asks a question about one limerick at a specific location.

I agree, I paid for Claude for a while. Even though they swear the context is huge and having a huge context uses up tokens like crack, it's near useless when source code in context just a few pages back. It was so frustrating as everything else was as good as anything and I liked the 'vibe'.

I used 4o last night and it was still perfectly aware of a C++ class I pasted 20 questions ago. I don't care about smart, I care about useful and this really contributes to the utility.

Post reply on HN