Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How outdated information hides in LLM token generation probabilities

blog.anj.ai

Post reply on HN
Print view View original

31–40 of 55 posts

Re: How outdated information hides in LLM token generation probabilities

#31

Post by DoctorOetker »

firtoz wrote: post #18

Looking towards the future, we will need to move away from "tokens are characters to print", We're kind of starting to consider this with "tool calls" but I believe an architectural shift will become necessary. We do have some kind of understanding of what kind of concept we want to emit next, e.g. ``` [The height:property name] of [Mount Bartle Frere:proper noun, describing an object to get a property out of], [in m…

Are there any minimal / micro LLM's that are paired with a large RAG vector database, but still have performance on par with the huge LLM's? i.e. shifting parameters from weights to vector database, so that a smaller LLM can fit in RAM, and the vector database on disk. Possibly multiple calls.

When you ask a human to switch context (changing topic) or to change activity (e.g. football to table tennis), they typically need some warm-up too, so it seems excessive to have all knowledge in high bandwidth RAM.

It would seem basic mathematics, set theory etc should stay in RAM.

Re: How outdated information hides in LLM token generation probabilities

#32

Post by ihsw »

aprilthird2021 wrote: post #30
0xKelsey wrote: post #2

> The scenario that I’m worried about, and that is playing out right now, is that they get good enough that we (or our leaders) become overconfident in their abilities and start integrating them into applications that they just aren’t ready for without a proper understanding of their limitations. Very true.

This is going to happen like in many engineering industries when a cheaper, more likely to be faulty part has been used in replacement of a more expensive, sturdier part. And people will groan but be unable to stop it, unfortunately

It is happening in insurance right now and it is an unmitigated disaster that nobody wants to address.

There is real data used for insurance premiums and claims payouts but it's being swapped out for AI slop, and the sales folks are getting bonuses for selling hot garbage and the executives are getting bonuses for buying hot garbage.

Re: How outdated information hides in LLM token generation probabilities

#33

Post by scarface_74 »

This problem is simple to solve for most real world use cases. Don’t trust any facts from an LLM and use your own trusted source of information and RAG where it will give you citations

https://chatgpt.com/share/6783df4c-904c-8010-a4b5-7301faea3b...

https://chatgpt.com/share/6783e0b8-ce78-8010-9177-d95eb77eac...

I use NotebookLM for most of my real world work these days with my project documentation.

Our company standard is GSuite and NotebookLM is specifically allowed.

Re: How outdated information hides in LLM token generation probabilities

#34

Post by scarface_74 »

croes wrote: post #15
choeger wrote: post #12

Earlier quoted context omitted.

As it turns out, correctness very often simply doesn't matter. Or not as much as one would intuitively think. How many shops are there optimizing "business strategies" with data that's -essentially- garbage?

For that LLMs are good but I bet some people want to use it for things where correctness is vital.

In that case you use RAG and have it tell you the source.

Re: How outdated information hides in LLM token generation probabilities

#35

Post by famouswaffles »

freehorse wrote: post #25
sd9 wrote: post #8

Earlier quoted context omitted.

I don’t think the public o1 can search the internet yet, unlike 4o. In principle it could know that something is more commonly cited based on its training data. But it could also just be hallucinating.

> it could know that something is more commonly cited based on its training data No there is no such concept or way to do something like that. LLMs do not have such kind of meta-knowledge over their training data or weights. But there could be explicit mentions about this on their training data and they could pick on that and that is probably the simplest explanation.

>LLMs do not have such kind of meta-knowledge over their training data or weights.

Not sure this is a claim that can be confidently made.

https://arxiv.org/abs/2309.00667

https://x.com/flowersslop/status/1873115669568311727?t=eBMbK...

Re: How outdated information hides in LLM token generation probabilities

#36

Post by Workaccount2 »

How would we rule out that the model didn't notice that the difference was small, and then simply put less weight into determining which is true.

I get the authors point, but I would have liked to see and example with a more egregious error.

Re: How outdated information hides in LLM token generation probabilities

#37

Post by ben_w »

fullstackwife wrote: post #17
ben_w wrote: post #11

Earlier quoted context omitted.

Same way as we can: learning which sources are more trustworthy. There's limits to how far you can go with this — not only do humans make mistakes with this, but even in the abstract theoretical it can never be perfect: https://en.wikipedia.org/wiki/Münchhausen_trilemma — but it is still the "how".

for the last 25+ years we rather not learned, but trusted the top3 of SERPs. Every ranking algorithm will be gamed eventually

I would say that we learned to trust the search engines; but otherwise I agree with you: every ranking algorithm will be gamed eventually.

(I wonder if giving an LLM content with intent to cause its users to spend money they didn't need to, would count as fraud, hacking, both, something else entirely?)

Re: How outdated information hides in LLM token generation probabilities

#38

Post by dotancohen »

blueflow wrote: post #27
HeatrayEnjoyer wrote: post #26

Earlier quoted context omitted.

Attention models learn what to pay attention to. It's been found that data that begin with "Wikipedia:" are automatically weighted higher by language models during training, completely unsupervised.

But this is the same problem - Wikipedia is a secondary source and should always get overruled by the primary source.

And there are documented cases of concentrated efforts to manipulate the content of Wikipedia.

Re: How outdated information hides in LLM token generation probabilities

#39

Post by dotancohen »

scarface_74 wrote: post #34
croes wrote: post #15

Earlier quoted context omitted.

For that LLMs are good but I bet some people want to use it for things where correctness is vital.

In that case you use RAG and have it tell you the source.

A RAG needs to be implemented by the LLM provider. The simple end user has no idea what that means, even though he will be (incorrectly) using the LLM for a vital purpose.

Re: How outdated information hides in LLM token generation probabilities

#40

Post by ffsm8 »

ihsw wrote: post #32
aprilthird2021 wrote: post #30

Earlier quoted context omitted.

This is going to happen like in many engineering industries when a cheaper, more likely to be faulty part has been used in replacement of a more expensive, sturdier part. And people will groan but be unable to stop it, unfortunately

It is happening in insurance right now and it is an unmitigated disaster that nobody wants to address. There is real data used for insurance premiums and claims payouts but it's being swapped out for AI slop, and the sales folks are getting bonuses for selling hot garbage and the executives are getting bonuses for buying hot garbage.

Huh, if sales uses AI/LLMs and succeeds with it, as it seemed to me youve implied with their bonuses ... Isn't that actually positive for the company?

Or do you mean they succeed by promising lies via AI?

Post reply on HN