Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How outdated information hides in LLM token generation probabilities

blog.anj.ai

Post reply on HN
Print view View original

11–20 of 55 posts

Re: How outdated information hides in LLM token generation probabilities

#11

Post by ben_w »

blueflow wrote: post #9
ascorbic wrote: post #3

The o1 example is interesting. In the CoT summary it acknowledges that the most recent official information is 1611m, but it then chooses to say 1622 because it's more commonly cited. It's like it over-thinks itself into the wrong answer.

How could a language model infer that the official information overrules anything else?

Same way as we can: learning which sources are more trustworthy.

There's limits to how far you can go with this — not only do humans make mistakes with this, but even in the abstract theoretical it can never be perfect: https://en.wikipedia.org/wiki/Münchhausen_trilemma — but it is still the "how".

Re: How outdated information hides in LLM token generation probabilities

#12

Post by choeger »

Terr_ wrote: post #7

> Welcome to the era of generative AI, where a mountain can have multiple heights, but also only one height, and the balance of my bank account gets to determine which one that is. All invisible to the end user and then rationalised away as a coincidence. I've always found the idea of untraceable, unfixable, unpredictable bugs in software... Offensive. Dirty. Unprofessional. So the last couple years have been been di…

As it turns out, correctness very often simply doesn't matter. Or not as much as one would intuitively think.

How many shops are there optimizing "business strategies" with data that's -essentially- garbage?

Re: How outdated information hides in LLM token generation probabilities

#13

Post by blueflow »

mistercow wrote: post #10
blueflow wrote: post #9

Earlier quoted context omitted.

How could a language model infer that the official information overrules anything else?

I’m not sure what kind of response you’re looking for, or if this is a rhetorical question or not. But “how could a language model infer…?” can be asked about a whole lot of things that language models have no problem reliably inferring.

> that language models have no problem reliably inferring

... the article did give me a different impression.

Re: How outdated information hides in LLM token generation probabilities

#14

Post by mistercow »

blueflow wrote: post #13
mistercow wrote: post #10

Earlier quoted context omitted.

I’m not sure what kind of response you’re looking for, or if this is a rhetorical question or not. But “how could a language model infer…?” can be asked about a whole lot of things that language models have no problem reliably inferring.

> that language models have no problem reliably inferring ... the article did give me a different impression.

I don’t think you read my comment correctly.

Re: How outdated information hides in LLM token generation probabilities

#15

Post by croes »

choeger wrote: post #12
Terr_ wrote: post #7

> Welcome to the era of generative AI, where a mountain can have multiple heights, but also only one height, and the balance of my bank account gets to determine which one that is. All invisible to the end user and then rationalised away as a coincidence. I've always found the idea of untraceable, unfixable, unpredictable bugs in software... Offensive. Dirty. Unprofessional. So the last couple years have been been di…

As it turns out, correctness very often simply doesn't matter. Or not as much as one would intuitively think. How many shops are there optimizing "business strategies" with data that's -essentially- garbage?

For that LLMs are good but I bet some people want to use it for things where correctness is vital.

Re: How outdated information hides in LLM token generation probabilities

#16

Post by delusional »

choeger wrote: post #12
Terr_ wrote: post #7

> Welcome to the era of generative AI, where a mountain can have multiple heights, but also only one height, and the balance of my bank account gets to determine which one that is. All invisible to the end user and then rationalised away as a coincidence. I've always found the idea of untraceable, unfixable, unpredictable bugs in software... Offensive. Dirty. Unprofessional. So the last couple years have been been di…

As it turns out, correctness very often simply doesn't matter. Or not as much as one would intuitively think. How many shops are there optimizing "business strategies" with data that's -essentially- garbage?

> How many shops are there optimizing "business strategies" with data that's -essentially- garbage?

How many of those shops are knowingly optimizing with garbage?

I'd argue that most of this data, which I would agree is garbage, is actually processed into seemingly good data through the complex and highly human process of self-deception and lies.

You don't tell the boss that the system you worked 2 month on is generating garbage, because then he'll replace your with someone who wouldn't tell him that. Instead you skirt evaluating it, even though you know better, and tell him that it's working fine. If the idiot chooses to do something stupid with your bad data, then that's his problem.

Re: How outdated information hides in LLM token generation probabilities

#17

Post by fullstackwife »

ben_w wrote: post #11
blueflow wrote: post #9

Earlier quoted context omitted.

How could a language model infer that the official information overrules anything else?

Same way as we can: learning which sources are more trustworthy. There's limits to how far you can go with this — not only do humans make mistakes with this, but even in the abstract theoretical it can never be perfect: https://en.wikipedia.org/wiki/Münchhausen_trilemma — but it is still the "how".

for the last 25+ years we rather not learned, but trusted the top3 of SERPs. Every ranking algorithm will be gamed eventually

Re: How outdated information hides in LLM token generation probabilities

#18

Post by firtoz »

Looking towards the future, we will need to move away from "tokens are characters to print", We're kind of starting to consider this with "tool calls" but I believe an architectural shift will become necessary.

We do have some kind of understanding of what kind of concept we want to emit next, e.g.

```

[The height:property name] of [Mount Bartle Frere:proper noun, describing an object to get a property out of], [in metres:attributes], is [?: retrieve value | (Mount Bartle Frere).("height", "metres")].

```

Re: How outdated information hides in LLM token generation probabilities

#20

Post by diggan »

sd9 wrote: post #8
freehorse wrote: post #4

Earlier quoted context omitted.

Does it search the internet for that? I assume so because else claiming how often something is cited does not make sense, but would be interesting to know surely. Even gpt4o mini with kagi gets it right with search enabled (and wrong without search enabled - tried over a few times to make sure).

I don’t think the public o1 can search the internet yet, unlike 4o. In principle it could know that something is more commonly cited based on its training data. But it could also just be hallucinating.

> In principle it could know that something is more commonly cited based on its training data

Could it? Without explicit training for that, how would it be expected to know it has to be able to count occurrences of something?

Post reply on HN