Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How outdated information hides in LLM token generation probabilities

blog.anj.ai

Post reply on HN
Print view View original

21–30 of 55 posts

Re: How outdated information hides in LLM token generation probabilities

#21

Post by giantrobot »

firtoz wrote: post #18

Looking towards the future, we will need to move away from "tokens are characters to print", We're kind of starting to consider this with "tool calls" but I believe an architectural shift will become necessary. We do have some kind of understanding of what kind of concept we want to emit next, e.g. ``` [The height:property name] of [Mount Bartle Frere:proper noun, describing an object to get a property out of], [in m…

Uh oh, that sounds suspiciously like querying structured data. You can't hype SQL or worse, SPARQL, to investors!

Re: How outdated information hides in LLM token generation probabilities

#23

Post by firtoz »

giantrobot wrote: post #21
firtoz wrote: post #18

Looking towards the future, we will need to move away from "tokens are characters to print", We're kind of starting to consider this with "tool calls" but I believe an architectural shift will become necessary. We do have some kind of understanding of what kind of concept we want to emit next, e.g. ``` [The height:property name] of [Mount Bartle Frere:proper noun, describing an object to get a property out of], [in m…

Uh oh, that sounds suspiciously like querying structured data. You can't hype SQL or worse, SPARQL, to investors!

Well, you can hype the results and then put it into a black box and call it a LLM anyway

Which is pretty much what O1 etc are

Update: it seems your recent submission[1] is pretty much that... interesting :D

1: https://github.com/caesarhq/textSQL

Re: How outdated information hides in LLM token generation probabilities

#24

Post by sd9 »

diggan wrote: post #20
sd9 wrote: post #8

Earlier quoted context omitted.

I don’t think the public o1 can search the internet yet, unlike 4o. In principle it could know that something is more commonly cited based on its training data. But it could also just be hallucinating.

> In principle it could know that something is more commonly cited based on its training data Could it? Without explicit training for that, how would it be expected to know it has to be able to count occurrences of something?

I think it would be more vibes based - commonly occurring things would be reinforced more in the weights. Rather than it explicitly counting the number of occurrences.

Re: How outdated information hides in LLM token generation probabilities

#25

Post by freehorse »

sd9 wrote: post #8
freehorse wrote: post #4

Earlier quoted context omitted.

Does it search the internet for that? I assume so because else claiming how often something is cited does not make sense, but would be interesting to know surely. Even gpt4o mini with kagi gets it right with search enabled (and wrong without search enabled - tried over a few times to make sure).

I don’t think the public o1 can search the internet yet, unlike 4o. In principle it could know that something is more commonly cited based on its training data. But it could also just be hallucinating.

> it could know that something is more commonly cited based on its training data

No there is no such concept or way to do something like that. LLMs do not have such kind of meta-knowledge over their training data or weights. But there could be explicit mentions about this on their training data and they could pick on that and that is probably the simplest explanation.

Re: How outdated information hides in LLM token generation probabilities

#26

Post by HeatrayEnjoyer »

blueflow wrote: post #9
ascorbic wrote: post #3

The o1 example is interesting. In the CoT summary it acknowledges that the most recent official information is 1611m, but it then chooses to say 1622 because it's more commonly cited. It's like it over-thinks itself into the wrong answer.

How could a language model infer that the official information overrules anything else?

Attention models learn what to pay attention to.

It's been found that data that begin with "Wikipedia:" are automatically weighted higher by language models during training, completely unsupervised.

Re: How outdated information hides in LLM token generation probabilities

#27

Post by blueflow »

HeatrayEnjoyer wrote: post #26
blueflow wrote: post #9

Earlier quoted context omitted.

How could a language model infer that the official information overrules anything else?

Attention models learn what to pay attention to. It's been found that data that begin with "Wikipedia:" are automatically weighted higher by language models during training, completely unsupervised.

But this is the same problem - Wikipedia is a secondary source and should always get overruled by the primary source.

Re: How outdated information hides in LLM token generation probabilities

#28

Post by diggan »

sd9 wrote: post #24
diggan wrote: post #20

Earlier quoted context omitted.

> In principle it could know that something is more commonly cited based on its training data Could it? Without explicit training for that, how would it be expected to know it has to be able to count occurrences of something?

I think it would be more vibes based - commonly occurring things would be reinforced more in the weights. Rather than it explicitly counting the number of occurrences.

So the probabilities would be skewed towards something, but unless the model could somehow count/infer its own weights, I don't see how it could "introspect" to see if something is more common than something else.

Re: How outdated information hides in LLM token generation probabilities

#29

Post by patrulek »

ascorbic wrote: post #3

The o1 example is interesting. In the CoT summary it acknowledges that the most recent official information is 1611m, but it then chooses to say 1622 because it's more commonly cited. It's like it over-thinks itself into the wrong answer.

I think i had similar case yesterday for Python script. It gave me code for older version of a module, but when i pasted the error i got, it corrected itself and gave me proper solution for version i had installed.

Re: How outdated information hides in LLM token generation probabilities

#30

Post by aprilthird2021 »

0xKelsey wrote: post #2

> The scenario that I’m worried about, and that is playing out right now, is that they get good enough that we (or our leaders) become overconfident in their abilities and start integrating them into applications that they just aren’t ready for without a proper understanding of their limitations. Very true.

This is going to happen like in many engineering industries when a cheaper, more likely to be faulty part has been used in replacement of a more expensive, sturdier part. And people will groan but be unable to stop it, unfortunately
Post reply on HN