Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Context Is Software, Weights Are Hardware

aravindjayendran.com

Post reply on HN
Print view View original

11–18 of 18 posts

Re: Context Is Software, Weights Are Hardware

#11

Post by adityaathalye »

maxaravind wrote: post #6
adityaathalye wrote: post #3

Earlier quoted context omitted.

> Let me know how you think about this. Well, I think of every Large Language Model as if it were a spectacularly faceted diamond. More on these lines in a recent-ish "thinking in public" attempt by yours truly, lay programmer, to interpret what an LLM-machine might be. Riff: LLMs are Software Diamonds https://www.evalapply.org/posts/llms-are-diamonds/

lol nice analogy. LLMs are frozen diamonds forged in compute. We need then to be malleable in production and change with experience.

Another way I see it is... Mind is process. LLM is (very lossy) snapshotted state of process/mind. LLM in-process is mind-emulator with potential to explore the state-space of the mind-snapshot. Consequently, and by its very construction, LLM cannot be mind.

Re: Context Is Software, Weights Are Hardware

#12

Post by qsera »

maxaravind wrote: post #9
qsera wrote: post #7

>for the sake of argument, that context can express everything weights can... Does this imply that a completely untrained model (random weights) should show intelligent behavior only by providing enough context?

Nope. Even if context can theoretically encode arbitrary computation under fixed weights, this requires the weights to implement a usable interpreter. Random weights almost surely do not. Training is what constructs that interpreter. Without it context has no meaningful computational semantics. It's kind of like asking if I make a random circuit with logic gates, does that become a universal computer that can run pro…

That was exactly what I was thinking. So it is a bit unclear why such a possibility should be even considered.

To be fair, I didn't really understand what idea this article is trying to get across..

Re: Context Is Software, Weights Are Hardware

#13

Post by maxaravind »

qsera wrote: post #12
maxaravind wrote: post #9

Earlier quoted context omitted.

Nope. Even if context can theoretically encode arbitrary computation under fixed weights, this requires the weights to implement a usable interpreter. Random weights almost surely do not. Training is what constructs that interpreter. Without it context has no meaningful computational semantics. It's kind of like asking if I make a random circuit with logic gates, does that become a universal computer that can run pro…

That was exactly what I was thinking. So it is a bit unclear why such a possibility should be even considered. To be fair, I didn't really understand what idea this article is trying to get across..

There has been a lot of talk about how continual learning might be "just and engineering challenge" and that we could have agents that continuously learn from experience by just having longer and longer context windows.

Here is a clip of Dario hinting at something similar: https://www.youtube.com/watch?v=Z0x99Uu4rJc

What I am trying to argue for in the article is how such a view might be misplaced - just extending the context length and adding more instructions in the context will not get you continual learning - the representational capacity of weights will be the limiting factor.

Just a fun way to think about it. Would love to hear your thoughts.

Re: Context Is Software, Weights Are Hardware

#14

Post by qsera »

maxaravind wrote: post #13
qsera wrote: post #12

Earlier quoted context omitted.

That was exactly what I was thinking. So it is a bit unclear why such a possibility should be even considered. To be fair, I didn't really understand what idea this article is trying to get across..

There has been a lot of talk about how continual learning might be "just and engineering challenge" and that we could have agents that continuously learn from experience by just having longer and longer context windows. Here is a clip of Dario hinting at something similar: https://www.youtube.com/watch?v=Z0x99Uu4rJc What I am trying to argue for in the article is how such a view might be misplaced - just extending th…

>just extending the context length and adding more instructions in the context will not get you continual learning...

I agree. But I am wondering if context would help in answering superficial questions and only fail when answering questions that require deeper understanding.

Re: Context Is Software, Weights Are Hardware

#16

Post by maxaravind »

qsera wrote: post #14
maxaravind wrote: post #13

Earlier quoted context omitted.

There has been a lot of talk about how continual learning might be "just and engineering challenge" and that we could have agents that continuously learn from experience by just having longer and longer context windows. Here is a clip of Dario hinting at something similar: https://www.youtube.com/watch?v=Z0x99Uu4rJc What I am trying to argue for in the article is how such a view might be misplaced - just extending th…

>just extending the context length and adding more instructions in the context will not get you continual learning... I agree. But I am wondering if context would help in answering superficial questions and only fail when answering questions that require deeper understanding.

I'd say the way to think about it is in terms of the questions you ask being in-distribution or out of distribution w.r.t the model training dataset.

Consider this, if something fundamental has changed in the world after the model was released(ie after the knowledge cut off date), then it would be very difficult for the model to reason about it. One concrete example is the the following: If you ask Opus or any decent coding model to do effort estimation on a coding task, then it would come up with multi week timelines - the models themselves doesn't know that because "they exist", these timelines have now been slashed to a few hours - you can try saying this in the prompt, however, they don't seem to internalise this.

Re: Context Is Software, Weights Are Hardware

#18

Post by qsera »

maxaravind wrote: post #16
qsera wrote: post #14

Earlier quoted context omitted.

>just extending the context length and adding more instructions in the context will not get you continual learning... I agree. But I am wondering if context would help in answering superficial questions and only fail when answering questions that require deeper understanding.

I'd say the way to think about it is in terms of the questions you ask being in-distribution or out of distribution w.r.t the model training dataset. Consider this, if something fundamental has changed in the world after the model was released(ie after the knowledge cut off date), then it would be very difficult for the model to reason about it. One concrete example is the the following: If you ask Opus or any decent…

So basically that is what I was saying.

Imagine an LLM that can also OCR. Would it be possible to make it OCR a totally new letter by only showing a single picture of it and including the fact in the context?

I think it would not be possible. That would be a good demonstration of the point I (and possibly you as well) is trying to get across.

Post reply on HN