Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning from context is harder than we thought

hy.tencent.com

Post reply on HN
Print view View original

121–130 of 140 posts

Re: Learning from context is harder than we thought

#121

Post by em-bee »

snayan wrote: post #106
em-bee wrote: post #96

Earlier quoted context omitted.

ok, fair point. what i am trying to say is that when we see/experience something that we can not describe we can create new words for it. we see something, we can name it. this directly contradicts the idea that language is the limit and that we can't talk about things that we don't have words for. that claim just doesn't make sense. the problem then is that these new words don't make any sense to anyone who doesn't…

Agreed, we can and will always come up with new words that attempt to approximate the experience, but, imo, they will always come up short. The abstracting inevitably leaves fidelity on the floor. It's necessary based on the way we're wired, struggle to think of a paradigm that would allow for the tribalism and connectedness that fostered human progress without shared verbal language initially, and written word later…

yes of course, language is by nature an abstraction, so by definition it will never describe the whole world perfectly, but it can describe it as well as we understand it. and the point that matters, once we have a shared experience we can name that experience, and between us it will then describe the full experience, whereas to bystanders it will be an abstraction.

language doesn't replace the actual experience. it isn't meant to. me living in china, and me telling you about my life in china are not the same thing, no matter how detailed my description. but that does not limit my experience. and if you lived in china too, then my description will refer your experience, and in that case the description will feel much more detailed.

the way i understand wittgensteins claim it not only suggests that language can't describe everything, which is only partly true, because it implies that language can not expand. it also means that i can not even experience what i can not describe, which makes even less sense. i can't feel cold because i have no word for it? huh?

(i feel like my argumentation jumps around or goes in circles, it doesn't feel well thought through. i hope it makes sense anyways. apologies for that.)

Re: Learning from context is harder than we thought

#122

Post by knollimar »

furyofantares wrote: post #67
cs702 wrote: post #24

The problem is even more fundamental: Today's models stop learning once they're deployed to production. There's pretraining, training, and finetuning, during which model parameters are updated. Then there's inference, during which the model is frozen. "In-context learning" doesn't update the model. We need models that keep on learning (updating their parameters) forever, online, all the time.

Why is learning an appropriate metaphor for changing weights but not for context? There are certainly major differences in what they are good or bad at and especially how much data you can feed them this way effectively. They both have plenty of properties we wish the other had. But they are both ways to take an artifact that behaves as if it doesn't know something and produce an artifact that behaves as if it does.…

Models gain information from context but probably not knowledge and definitely not wisdom.

Re: Learning from context is harder than we thought

#123

Post by wat10000 »

furyofantares wrote: post #67
cs702 wrote: post #24

The problem is even more fundamental: Today's models stop learning once they're deployed to production. There's pretraining, training, and finetuning, during which model parameters are updated. Then there's inference, during which the model is frozen. "In-context learning" doesn't update the model. We need models that keep on learning (updating their parameters) forever, online, all the time.

Why is learning an appropriate metaphor for changing weights but not for context? There are certainly major differences in what they are good or bad at and especially how much data you can feed them this way effectively. They both have plenty of properties we wish the other had. But they are both ways to take an artifact that behaves as if it doesn't know something and produce an artifact that behaves as if it does.…

It’s not very good in context, for one thing. Context isn’t that big, and RAG is clumsy. Working with an LLM agent is like working with someone who can’t form new long term memories. You have to get them up to speed from scratch every time. You can accelerate this by putting important stuff into the context, but that slows things down and can’t handle very much stuff.

Re: Learning from context is harder than we thought

#124

Post by YeGoblynQueenne »

>> Current language models do not handle context this way. They rely primarily on parametric knowledge—information compressed into their weights during massive pre-training runs. At inference time, they function largely by recalling this static, internal memory, rather than actively learning from new information provided in the moment.

>> This creates a structural mismatch. We have optimized models to excel at reasoning over what they already know yet users need them to solve tasks that depend on messy, constantly evolving context. We built models that rely on what they know from the past, but we need context learners that rely on what they can absorb from the environment in the moment.

>> To bridge this gap, we must fundamentally change our optimization direction.

All this is right and what critics of the deafening over-hyping of LLMs have long pointed out.

So what do the authors propose we do? Currently, they propose ... a benchmark. But, what's that going to achieve? We know very well that LLMs, neural nets in general, are masters at saturating benchmarks without actually mastering the abilities that the benchmarks are meant to be measuring.

What happens if in a year or so, as will definite happen, LLMs saturate this benchmark too? Will we all have to agree that LLMs can now do "context learning" and then move on to the next big thing? This year it's "world models", last year it was "reasoning" and next year it's going to be "context learning"? And then, what? Where is this all leading to, if after all the billions spent and all the benchmarks beaten conclusively, LLMs still can't do reasoning, can't do world-modelling, can't do context learning and so on, and so forth?

Re: Learning from context is harder than we thought

#125

Post by furyofantares »

wat10000 wrote: post #123
furyofantares wrote: post #67

Earlier quoted context omitted.

Why is learning an appropriate metaphor for changing weights but not for context? There are certainly major differences in what they are good or bad at and especially how much data you can feed them this way effectively. They both have plenty of properties we wish the other had. But they are both ways to take an artifact that behaves as if it doesn't know something and produce an artifact that behaves as if it does.…

It’s not very good in context, for one thing. Context isn’t that big, and RAG is clumsy. Working with an LLM agent is like working with someone who can’t form new long term memories. You have to get them up to speed from scratch every time. You can accelerate this by putting important stuff into the context, but that slows things down and can’t handle very much stuff.

The article does demonstrate how bad it is in context.

Context has a lot of big advantages over training though, too, it's not one-sided. Upfront cost and time are the big obvious ones, but context also works better than training on small amounts of data, and it's easier to delete or modify.

Like even for a big product like Claude Code from someone that controls the model, although I'm sure they do a lot of training to make the product better, they're not gonna just rely entirely on training and go with a nearly blank system prompt.

Re: Learning from context is harder than we thought

#126

Post by ausbah »

yunohn wrote: post #104
TZubiri wrote: post #13

This is quite on brand for China. I think they are experts at reverse engineering and learning 'from context' rather than by formal consumption of foreign training material. The fictional training data with a made up country and laws was a very interesting experiment design, I can imagine that's how they approach making business with other countries. Like an alien made up system they have to learn on the spot.

> experts at reverse engineering and learning 'from context' rather than by formal consumption of foreign training material China (as with other Asian cultures like India) is well known for their schooling involving extreme amounts of formal training material consumption. The reverse-engineering is performed with a solid foundation of theoretical understanding.

everytime someone tries to split the hair on how China is pulling ahead on most important metrics reads like coping of the nth degree

Re: Learning from context is harder than we thought

#127

Post by jmalicki »

YeGoblynQueenne wrote: post #124

>> Current language models do not handle context this way. They rely primarily on parametric knowledge—information compressed into their weights during massive pre-training runs. At inference time, they function largely by recalling this static, internal memory, rather than actively learning from new information provided in the moment. >> This creates a structural mismatch. We have optimized models to excel at reason…

> Where is this all leading to, if after all the billions spent and all the benchmarks beaten conclusively, LLMs still can't do reasoning, can't do world-modelling, can't do context learning and so on, and so forth?

Humans completely displaced from the workforce while they harp "but LLMs can't really think and don't really have creativity!"

Re: Learning from context is harder than we thought

#128

Post by shimman »

jmalicki wrote: post #127
YeGoblynQueenne wrote: post #124

>> Current language models do not handle context this way. They rely primarily on parametric knowledge—information compressed into their weights during massive pre-training runs. At inference time, they function largely by recalling this static, internal memory, rather than actively learning from new information provided in the moment. >> This creates a structural mismatch. We have optimized models to excel at reason…

> Where is this all leading to, if after all the billions spent and all the benchmarks beaten conclusively, LLMs still can't do reasoning, can't do world-modelling, can't do context learning and so on, and so forth? Humans completely displaced from the workforce while they harp "but LLMs can't really think and don't really have creativity!"

Humans are being displaced because moronic business magnates are trying to force feed the country their wares but failing spectacularly so now they are forcing governments across the world to buy their wares under threat of the US government.

Re: Learning from context is harder than we thought

#129

Post by smolder »

cs702 wrote: post #24

The problem is even more fundamental: Today's models stop learning once they're deployed to production. There's pretraining, training, and finetuning, during which model parameters are updated. Then there's inference, during which the model is frozen. "In-context learning" doesn't update the model. We need models that keep on learning (updating their parameters) forever, online, all the time.

We need models that are smarter than humans. So far, the cost of an AI query + training is dwarfing the effort it would take to teach an intelligent human how to do a task. We are dumping an incredibly amount of money/effort into making AI do stuff when it's still not competitive with humans, because dumbass people are controlling investment. The stock market is not a replacement for competent investment. The fact people buy meme coins shows how fucked we are.

Deceiving people is not a sustainable business model, but it is the most prominent one in the US right now. Lie to the public, sell them stuff that's bad for them at too high of a price, get rich quick, then act confused when your economy collapses because the victims of your grift can't spend anymore.

Re: Learning from context is harder than we thought

#130

Post by snayan »

em-bee wrote: post #121
snayan wrote: post #106

Earlier quoted context omitted.

Agreed, we can and will always come up with new words that attempt to approximate the experience, but, imo, they will always come up short. The abstracting inevitably leaves fidelity on the floor. It's necessary based on the way we're wired, struggle to think of a paradigm that would allow for the tribalism and connectedness that fostered human progress without shared verbal language initially, and written word later…

yes of course, language is by nature an abstraction, so by definition it will never describe the whole world perfectly, but it can describe it as well as we understand it. and the point that matters, once we have a shared experience we can name that experience, and between us it will then describe the full experience, whereas to bystanders it will be an abstraction. language doesn't replace the actual experience. it…

Na, your argument makes sense. Loving this discussion.

Ok, so I don't agree that it implies language cannot expand. I believe it's a bit more nuanced than that, I believe what he's trying to say is that it cannot expand sufficiently to truly capture the experience. We will inevitably dumb it down or lose fidelity or whatever. The 'unsayables' as he called them, I believe he felt he was trying to protect their integrity by saying we should not attempt to distill them down to words.

As for the I cannot experience what I cannot describe... I agree with this statement deeply. Well, I think it's a function of ego or whatever you want to call it. We go through life and are shaped by our experiences. As we continue to experience life, we have more and more beliefs bouncing around in our head as a function of more experience. Ahhh, this just happened, it's like when I did X, etc etc. As we get older we get more and more bogged down by these limiting beliefs until everything we experience is going through our personal interpretive filter rather than just being experienced for what it is.

It's the Buddhist idea of the finger pointing at the moon. Don't mistake the finger(thoughts, words, etc) for the moon (the direct experience).

Well, that's been my personal experience, until I started looking inside and poking around at my belief structure, I had noooo idea how much my interpretation of the world had been shaped by prior lived experience, personally, and societally.

In your cold example... If you had no word for it, I believe most people would end up using the closest approximation out of the words they do know effectively blinding themselves to the reality of this new/unique experience for them. How though, would someone know, ahh there is no word for this, lets expand the language.

Gotta embrace not knowing/the beginners mind, and in my personal experience this is a process of subtraction rather than addition.

Post reply on HN