Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Microeconomics of Artificial Intelligence (2025)

direct.mit.edu

Post reply on HN
Print view View original

41–50 of 52 posts

Re: The Microeconomics of Artificial Intelligence (2025)

#41

Post by howunfortunate »

imtringued wrote: post #28
howunfortunate wrote: post #15

Earlier quoted context omitted.

That's not quite true though. The fact that most models are in practice non-deterministic has been a huge point of contention. And there's nothing inherently stopping labs from continuously fine-tuning the weights after every new invocation. It's just a difficult (not to mention expensive) software and ML problem.

It's not "not quite true", it's literally true because alternative architectures like RNNs and Mamba fully update their own internal states, whereas transformers only append to the context. RNNs and Mamaba do not update their weights, but you could hypothetically scale the internal state to be as big as Fable's and GPT 6's parameters.

At least one mechanism to update transformers' internal states already exists, there is nothing stopping anyone from performing backprop after every session.

It just has big technical and economic challenges. But I expect advances there. There have actually already been big advances, though done in bulk fashion (RLHF).

Re: The Microeconomics of Artificial Intelligence (2025)

#42

Post by howunfortunate »

imtringued wrote: post #29
howunfortunate wrote: post #22

Earlier quoted context omitted.

Yes, catastrophic forgetting is absolutely one of the problems that needs to be solved to enable something like this. My broader point is just that there's nothing inherent to the structure of LLMs that stops them from updating their weights and continuously learning from environmental feedback in the way humans do, and there's already solid templates for how they could push even further in that direction. But as an…

>My broader point is just that there's nothing inherent to the structure of LLMs that stops them from updating their weights and continuously learning from environmental feedback in the way humans do, and there's already solid templates for how they could push even further in that direction. "LLM" is a branded model as a product. Of course it could be anything, as long as it fulfills the product category. But we live…

Ok just say "transformer" then.

What can a transformer not do that people say they can do?

The parent comment said, paraphrasing, "learn from interaction with the world", and I'm responding, they absolutely can already do this by taking their logs of interaction with humans and updating their weights through backprop.

The reason you don't see that done "live" is primarily an economics problem rather than a limitation of the model structure.

Re: The Microeconomics of Artificial Intelligence (2025)

#43

Post by howunfortunate »

nairboon wrote: post #33
howunfortunate wrote: post #13

Earlier quoted context omitted.

I think that's relatively emergent too though! BERT never really did that (at least to my recollection), presumably because its training was never sufficient for it to develop corrective reasoning in a chain of thought.

What exactly do you mean with "emergent" here?

In a general sense I mean something like "arising through complex interaction between parts", like the murmurations of certain birds or unexpected mechanics found in physics sandbox games.

In this specific instance I mean I don't think it was obvious given the structure of the model, it only "emerged" when we applied massive scale, which allowed for unexpected interactions to happen in the deep networks involved.

Re: The Microeconomics of Artificial Intelligence (2025)

#44

Post by [deleted] »

redwood wrote: post #40
nightski wrote: post #19

Earlier quoted context omitted.

I was not saying that they are deterministic, rather that the distributions (aka weights) are fixed. A model as deployed today at anthropic/open ai/etc is not learning beyond the context as far as I know. What prevents continuous fine-tuning from what I understand is catastrophic forgetting. You can do things like RLHF which are built to minimize the damage but that is more about bringing out capabilities of the base…

But they routinely leverage web search in connection to providing responses which means they're leveraging their static intelligence on top of a dynamic context corpus. This is critically important and arguably similar to humans in other words the typical human might take a while to develop a new skill but they can change how they dynamically leverage an existing skill based on context

[deleted]

Re: The Microeconomics of Artificial Intelligence (2025)

#45

Post by nightski »

howunfortunate wrote: post #42
imtringued wrote: post #29

Earlier quoted context omitted.

>My broader point is just that there's nothing inherent to the structure of LLMs that stops them from updating their weights and continuously learning from environmental feedback in the way humans do, and there's already solid templates for how they could push even further in that direction. "LLM" is a branded model as a product. Of course it could be anything, as long as it fulfills the product category. But we live…

Ok just say "transformer" then. What can a transformer not do that people say they can do? The parent comment said, paraphrasing, "learn from interaction with the world", and I'm responding, they absolutely can already do this by taking their logs of interaction with humans and updating their weights through backprop. The reason you don't see that done "live" is primarily an economics problem rather than a limitation…

Alright let's assume your premise is true, that transformers can learn from interaction with the world by updating their weights - then why isn't this done?

Because backprop fundamentally wants the entire data set in every pass. It doesn't behave well and is destructive when you update after the pre-training phase. RLHF/LORA are attempts to work around that and effective at what they do, but it is not learning in the sense you are talking about and also do not fully address the catastrophic forgetting problem. This architecture as is - is not compatible with continual learning.

Re: The Microeconomics of Artificial Intelligence (2025)

#46

Post by howunfortunate »

nightski wrote: post #45
howunfortunate wrote: post #42

Earlier quoted context omitted.

Ok just say "transformer" then. What can a transformer not do that people say they can do? The parent comment said, paraphrasing, "learn from interaction with the world", and I'm responding, they absolutely can already do this by taking their logs of interaction with humans and updating their weights through backprop. The reason you don't see that done "live" is primarily an economics problem rather than a limitation…

Alright let's assume your premise is true, that transformers can learn from interaction with the world by updating their weights - then why isn't this done? Because backprop fundamentally wants the entire data set in every pass. It doesn't behave well and is destructive when you update after the pre-training phase. RLHF/LORA are attempts to work around that and effective at what they do, but it is not learning in the…

> why isn't this done

It IS done with every new model release. Why do you think SpaceXAI bought Cursor? And then immediately had a huge jump in capability with their next model version?

It's just done in large batches for economic reasons.

> backprop fundamentally wants the entire data set in every pass

I'm genuinely unsure what you mean, it's not even possible to run backprop in this way?

Re: The Microeconomics of Artificial Intelligence (2025)

#49

Post by pixl97 »

galaxyLogic wrote: post #35
howunfortunate wrote: post #22

Earlier quoted context omitted.

Yes, catastrophic forgetting is absolutely one of the problems that needs to be solved to enable something like this. My broader point is just that there's nothing inherent to the structure of LLMs that stops them from updating their weights and continuously learning from environmental feedback in the way humans do, and there's already solid templates for how they could push even further in that direction. But as an…

LLMs are "readonly" I guess for several reasons: 1. Technical cost of updating the mode. 2. Inability to trust every user's "truth". 3. Ability of AGENT-HARNESSES to learn with the help of the human user. So agents learn, LLM already knows everything it will ever know, and ESPECIALLY it has already learned how to understand human language. No 3. above means there is no danger of the LLM getting corrupted. But the age…

4. Inability to ensure models don't go off the fricken rails and become skynet.

Re: The Microeconomics of Artificial Intelligence (2025)

#50

Post by nightski »

howunfortunate wrote: post #46
nightski wrote: post #45

Earlier quoted context omitted.

Alright let's assume your premise is true, that transformers can learn from interaction with the world by updating their weights - then why isn't this done? Because backprop fundamentally wants the entire data set in every pass. It doesn't behave well and is destructive when you update after the pre-training phase. RLHF/LORA are attempts to work around that and effective at what they do, but it is not learning in the…

> why isn't this done It IS done with every new model release. Why do you think SpaceXAI bought Cursor? And then immediately had a huge jump in capability with their next model version? It's just done in large batches for economic reasons. > backprop fundamentally wants the entire data set in every pass I'm genuinely unsure what you mean, it's not even possible to run backprop in this way?

No, I'd have to guess that new model releases have either new from scratch or continued pre-training. This is not the same as continual learning. Starting a new pre-training session is a dramatically different affair and involves utilizing the entire source data set in some fashion. Not just continued training on new data. At least from my understanding.

Bolting on new data to an existing model (fine-tuning) is precisely what gets you catastrophic forgetting.

Post reply on HN