Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning from context is harder than we thought

hy.tencent.com

Post reply on HN
Print view View original

131–140 of 140 posts

Re: Learning from context is harder than we thought

#131

Post by em-bee »

snayan wrote: post #130
em-bee wrote: post #121

Earlier quoted context omitted.

yes of course, language is by nature an abstraction, so by definition it will never describe the whole world perfectly, but it can describe it as well as we understand it. and the point that matters, once we have a shared experience we can name that experience, and between us it will then describe the full experience, whereas to bystanders it will be an abstraction. language doesn't replace the actual experience. it…

Na, your argument makes sense. Loving this discussion. Ok, so I don't agree that it implies language cannot expand. I believe it's a bit more nuanced than that, I believe what he's trying to say is that it cannot expand sufficiently to truly capture the experience. We will inevitably dumb it down or lose fidelity or whatever. The 'unsayables' as he called them, I believe he felt he was trying to protect their integri…

I had noooo idea how much my interpretation of the world had been shaped by prior lived experience

this is an interesting point. it's very true of course. there is probably some philosophical or biological explanation for this, something about optimization, because interpreting every situation from first principles takes to much effort. living in a different culture is one way to teach you to look at things differently.

but i think it is an issue independent of language. the problem is not lack of ability to describe the experience, but mistakenly using an already familiar abstraction to describe a new experience. but that's not the end of it, because repeatedly making that experience eventually helps you realize that the description you used is wrong, and you adjust to create a better description.

actually a better example than cold is the word umami. in our languages we have terms for sweet, sour, salty and bitter. turns out our body has dedicated receptors for umami, but we were not aware of that, and we never named it. even today it still feels like a foreign concept, but we have evidence that it is a real biological experience and not just a cultural idea.

the thing, is we certainly experienced umami in some way, but we could not talk about it, we were not consciously aware of it. and we still are not. i can tell very sweet from somewhat sweet to not sweet at all, but what's very umami or not umami? how does that even work? there is a whole dimension of language that our culture is missing. but, it's a cultural problem, not a linguistic one. because now we do have a word for it. and still, at least i struggle with the concept.

interestingly i think this example shows how humans learn from context. our (western) culture is missing the context for umami. we need to build up that context to allow others to learn about it.

Re: Learning from context is harder than we thought

#132

Post by snovv_crash »

Legend2440 wrote: post #113
snovv_crash wrote: post #111

Earlier quoted context omitted.

Yes, that's basically the point. You get 'free' continuous learning just by throwing the new data into the pool. Needing an explicit training step is a weakness that makes CL hard to make work for many other approaches. For any practical application KNN will need some kind of accelerated search structure (eg Kd-tree for < ~7 dimensions) which then requires support for dynamic insertions. But this is an engineering pr…

>Needing an explicit training step is a weakness that makes CL hard to make work for many other approaches. On the other hand, not having an explicit training step is a huge weakness of KNN. Training-based methods scale better because the storage and runtime requirements are independent of dataset size. You can compress 100TB of training data down into a 70GB LLM. A KNN on the same data would require keeping around t…

Feature engineering is a thing, you don't need the full data source for KNN to do the search in. It is already used extensively in RAG type lookup systems, for example.

Re: Learning from context is harder than we thought

#133

Post by davidguetta »

thesz wrote: post #107
DoctorOetker wrote: post #85

Earlier quoted context omitted.

will catastrophic forgetting still occur if a fraction of the update sentences are the original training corpus? is the real issue actually catastrophic forgetting or overfitting? nothing prevents users from continuing the learning as they use a model

Catastrophic forgetting is overfitting.

not exactly, not at all even in term of the way the llm are trained.

In RL it can be that you are not getting meaningful data anymore because you are 'too good' and dont get anymore the "this is a bad answer" signal so you can't estimate the gradient.

Re: Learning from context is harder than we thought

#134

Post by nxobject »

cs702 wrote: post #24

The problem is even more fundamental: Today's models stop learning once they're deployed to production. There's pretraining, training, and finetuning, during which model parameters are updated. Then there's inference, during which the model is frozen. "In-context learning" doesn't update the model. We need models that keep on learning (updating their parameters) forever, online, all the time.

I wish that agents could “sleep and consolidate” like humans do.

Re: Learning from context is harder than we thought

#135

Post by nxobject »

BobbyTables2 wrote: post #69
cs702 wrote: post #24

The problem is even more fundamental: Today's models stop learning once they're deployed to production. There's pretraining, training, and finetuning, during which model parameters are updated. Then there's inference, during which the model is frozen. "In-context learning" doesn't update the model. We need models that keep on learning (updating their parameters) forever, online, all the time.

How long will it take someone to poison such a model by teaching it wrong things? Even humans fall for propaganda repeated over and over . The current non-learning model is unintentionally right up there with the “immutable system” and “infrastructure as code” philosophy.

> The current non-learning model is unintentionally right up there with the “immutable system” and “infrastructure as code” philosophy.

As long as training material remains the proprietary secret sauce, the average user doesn’t already see or benefit from that - it’s all a promise and a black box to us.

Re: Learning from context is harder than we thought

#136

Post by nxobject »

jmalicki wrote: post #127
YeGoblynQueenne wrote: post #124

>> Current language models do not handle context this way. They rely primarily on parametric knowledge—information compressed into their weights during massive pre-training runs. At inference time, they function largely by recalling this static, internal memory, rather than actively learning from new information provided in the moment. >> This creates a structural mismatch. We have optimized models to excel at reason…

> Where is this all leading to, if after all the billions spent and all the benchmarks beaten conclusively, LLMs still can't do reasoning, can't do world-modelling, can't do context learning and so on, and so forth? Humans completely displaced from the workforce while they harp "but LLMs can't really think and don't really have creativity!"

I think that’s more of a reflection on what employers are willing to pay for…

Re: Learning from context is harder than we thought

#137

Post by jmalicki »

nxobject wrote: post #136
jmalicki wrote: post #127

Earlier quoted context omitted.

> Where is this all leading to, if after all the billions spent and all the benchmarks beaten conclusively, LLMs still can't do reasoning, can't do world-modelling, can't do context learning and so on, and so forth? Humans completely displaced from the workforce while they harp "but LLMs can't really think and don't really have creativity!"

I think that’s more of a reflection on what employers are willing to pay for…

Agreed, but does it matter? LLMs will affect us by how much they displace the parts of us employers are willing to pay for. The "but they don't really think!" is just a cope.

Re: Learning from context is harder than we thought

#138

Post by carlmr »

fhd2 wrote: post #88
furyofantares wrote: post #67

Earlier quoted context omitted.

Why is learning an appropriate metaphor for changing weights but not for context? There are certainly major differences in what they are good or bad at and especially how much data you can feed them this way effectively. They both have plenty of properties we wish the other had. But they are both ways to take an artifact that behaves as if it doesn't know something and produce an artifact that behaves as if it does.…

I suppose ultimately, the external behaviour of the system is what matters. You can see the LLM as the system, on a low level, or even the entire organisation of e.g. OpenAI at a high level. If it's the former: Yeah, I'd argue they don't "learn" much (!) past inference. I'd find it hard to argue context isn't learning at all. It's just pretty limited in how much can be learned post inference. If you look at the entir…

To stretch the human analogy, it's short term memory that's completely disconnected from long term memory.

The models currently have anteretrograde amnesia.

Re: Learning from context is harder than we thought

#139

Post by bluegatty »

derefr wrote: post #109
bluegatty wrote: post #61

Earlier quoted context omitted.

That's not how training works - adjusting model weights to memorize a single data item is not going to fly. Model weights store abilities, not facts - generally. Unless the fact is very widely used and widely known, with a ton of context around it. The model can learn the day JFK died because there are millions of sparse examples of how that information exists in the world, but when you're working on a problem, you m…

> That's not how training works - adjusting model weights to memorize a single data item is not going to fly. Apologies; I think I got us all kind of off-track in this comment thread by stretching the definition of the term "fine-tuning" in my ancestor comment above. Actual fine-tuning of the base model's weights (as one would do to customize a base model into a domain-specific model) works the way you're talking abo…

This is a fine thought, I'm reluctant about it. It could work, I don't think it's obvious. It's very, very hard to know what to train for and not, and this still leaves the 'fact v. skill' problem - even LORA won't enable a model to remember your favourite lunch place.

This is kind of an existential problem with context I think. Maybe we need new architectures.

Re: Learning from context is harder than we thought

#140

Post by pankajdoharey »

thesz wrote: post #107
DoctorOetker wrote: post #85

Earlier quoted context omitted.

will catastrophic forgetting still occur if a fraction of the update sentences are the original training corpus? is the real issue actually catastrophic forgetting or overfitting? nothing prevents users from continuing the learning as they use a model

Catastrophic forgetting is overfitting.

No, it’s actually the math of overwriting. Imagine you hiked down into a valley Task A and settled there. Then, you decide to climb a new mountain to find a different valley Task B. You successfully move to the new valley, but in doing so, you destroy the path back to the first one. You are now stuck in the new valley and have completely 'forgotten' how to get back to the first one.
Post reply on HN