Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large models of what? Mistaking engineering achievements for linguistic agency

arxiv.org

Post reply on HN
Print view View original

11–20 of 162 posts

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#11

Post by abernard1 »

GeneralMayhem wrote: post #10
abernard1 wrote: post #9

Earlier quoted context omitted.

> LLMs do not have corporeal experience. But it's not obvious that this means that they cannot, a priori, have an "internal" concept of reality, or that it's impossible to gain such an understanding from text. I would argue it is (obviously) impossible the way the current implementation of models work. How could a system which produces a single next word based upon a likelihood and and a parameter called a "temperatu…

The argument would be that that conceptual model is encoded in the intermediate-layer parameters of the model, in a different but analogous way to how it's encoded in the graph and chemical structure of your neurons.

I agree that's an argument. I would contend that argument is obviously false. If it were true, LLMs could multiply scalar numbers together trivially. It should be the easiest thing in the world for them. The network required to do that well is extremely small, the parameter sizes of these models are gigantic, and the textual expression is highly regular: multiplication is the simplest concept imaginable.

That they cannot do that basic task implies to me that they have almost no conceptual understanding unless the fit is almost memorizable or the space is highly regular. That LLMs can't multiply numbers properly isn't surprising if they don't really understand concepts prior to emitting text. Where they do logical tasks, that can be done with minimal or no understanding, because syllogisms and logical formalisms are highly structured in text arguments.

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#12

Post by Davidzheng »

abernard1 wrote: post #9
GeneralMayhem wrote: post #8

I am highly skeptical of LLMs as a mechanism to achieve AGI, but I also find this paper fairly unconvincing, bordering on tautological. I feel similarly about this as to what I've read of Chalmers - I agree with pretty much all of the conclusions, but I don't feel like the text would convince me of those conclusions if I disagreed; it's more like it's showing me ways of explaining or illustrating what I already belie…

> LLMs do not have corporeal experience. But it's not obvious that this means that they cannot, a priori, have an "internal" concept of reality, or that it's impossible to gain such an understanding from text. I would argue it is (obviously) impossible the way the current implementation of models work. How could a system which produces a single next word based upon a likelihood and and a parameter called a "temperatu…

It's only because you can essentially put the llms in a simulations that you can have this argument. We can imagine the human brain also in a simulation which we can replay over and over again and adjust various parameters of the physical brain to change the temperature. These sort of arguments can never distinguish between llm and humans.

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#13

Post by gwervc »

abernard1 wrote: post #9
GeneralMayhem wrote: post #8

I am highly skeptical of LLMs as a mechanism to achieve AGI, but I also find this paper fairly unconvincing, bordering on tautological. I feel similarly about this as to what I've read of Chalmers - I agree with pretty much all of the conclusions, but I don't feel like the text would convince me of those conclusions if I disagreed; it's more like it's showing me ways of explaining or illustrating what I already belie…

> LLMs do not have corporeal experience. But it's not obvious that this means that they cannot, a priori, have an "internal" concept of reality, or that it's impossible to gain such an understanding from text. I would argue it is (obviously) impossible the way the current implementation of models work. How could a system which produces a single next word based upon a likelihood and and a parameter called a "temperatu…

> generative systems are basically always hallucinating: they do not have any concepts at all. That they are "right" as often as they are is a testament to the power of curve fitting and compression of basis functions in high dimensionality spaces

It's refreshing to read someone who "got it". Sad that before my upvote the comment was grayed out.

Any proponent of conceptual or other wishful/magical thinking shoud come with proofs, since it is the hypothesis that diverge from the definition of a LLM.

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#14

Post by Lerc »

GeneralMayhem wrote: post #8

I am highly skeptical of LLMs as a mechanism to achieve AGI, but I also find this paper fairly unconvincing, bordering on tautological. I feel similarly about this as to what I've read of Chalmers - I agree with pretty much all of the conclusions, but I don't feel like the text would convince me of those conclusions if I disagreed; it's more like it's showing me ways of explaining or illustrating what I already belie…

>that sufficiently advanced mimicry is not only indistinguishable from the real thing, but at the limit in fact is the real thing.

While sufficiently does a lot of the heavy lifting here, the indistinguishable criteria implicitly means there must be no-way to tell if it is not the real thing. The belief that it is the real thing comes from the intuition that anything that can be everything a person must be, but have that fundamental essence of being a person. I don't think people could really conceive an alternative without resorting to prejudice which they could equally apply to machines or people.

I take the arguments such as in this paper to be instead making the claim that because X cannot be Y you will never be able to make X indistinguishable from Y. It is more a prediction of future failure than a judgment on an existing thing.

I end up looking at some of these complaints from the point of view of my sometimes profession of Game Developer. When I show someone a game in development to playtest they will find a bunch of issues. The vast majority of those issues, not only am I already aware of, but I have a much more detailed perspective of what the problem is and how it might be fixed. I have been seeing the problem, over and over, every day as I work. The problem persists because there are other things to do before fixing the issue, some of which might render the issue redundant anyway.

I feel like a lot of the criticisms of AI are like this they are like the playtesters pointing out issues in the current state where those working on the problems are generally well aware of particular issues and have a variety of solutions in mind that might help.

Clear statements of deficiencies in ability are helpful as a guide to measure future success.

I'm also in the camp that LLM's cannot be an AGI on its own, on the other hand I do think the architecture might be extended to become one. There is an easy out for any criticism to say, "Well, it's not an LLM anymore".

In a way that ends up with a lot of people saying

.The current models cannot do the things we know the current models cannot do

.Future models will not be able to do those things if they are the same as the current ones

.Therefore the things that will be able to do those things will be different

That is true, but hardly enlightening.

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#16

Post by GaggiX »

abernard1 wrote: post #11
GeneralMayhem wrote: post #10

Earlier quoted context omitted.

The argument would be that that conceptual model is encoded in the intermediate-layer parameters of the model, in a different but analogous way to how it's encoded in the graph and chemical structure of your neurons.

I agree that's an argument. I would contend that argument is obviously false. If it were true, LLMs could multiply scalar numbers together trivially. It should be the easiest thing in the world for them. The network required to do that well is extremely small, the parameter sizes of these models are gigantic, and the textual expression is highly regular: multiplication is the simplest concept imaginable. That they ca…

Multiplication requires O(n^2) complexity with the usual algorithm used by humans, LLMs have a constant amount of computation available and they are not really efficient machines for math evaluation. They can definitely evaluate unseen expressions and you train a neural network to learn how to do sums and multiplications, I have trained models on sums and they are able to do sums never seen during training, the model learns the algorithm just by giving it inputs and outputs.

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#17

Post by jdietrich »

abernard1 wrote: post #11
GeneralMayhem wrote: post #10

Earlier quoted context omitted.

The argument would be that that conceptual model is encoded in the intermediate-layer parameters of the model, in a different but analogous way to how it's encoded in the graph and chemical structure of your neurons.

I agree that's an argument. I would contend that argument is obviously false. If it were true, LLMs could multiply scalar numbers together trivially. It should be the easiest thing in the world for them. The network required to do that well is extremely small, the parameter sizes of these models are gigantic, and the textual expression is highly regular: multiplication is the simplest concept imaginable. That they ca…

LLMs do contain conceptual representations and LLMs are capable of abstract reasoning. This is trivially provable by asking them to reason about something that is a) purely abstract and b) not in the training data, e.g. "All floots are gronks. Some gronks are klorps. Are any floots klorps?" Any of the leading LLMs will correctly answer questions of this type much more often than chance.

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#18

Post by [deleted] »

abernard1 wrote: post #11
GeneralMayhem wrote: post #10

Earlier quoted context omitted.

The argument would be that that conceptual model is encoded in the intermediate-layer parameters of the model, in a different but analogous way to how it's encoded in the graph and chemical structure of your neurons.

I agree that's an argument. I would contend that argument is obviously false. If it were true, LLMs could multiply scalar numbers together trivially. It should be the easiest thing in the world for them. The network required to do that well is extremely small, the parameter sizes of these models are gigantic, and the textual expression is highly regular: multiplication is the simplest concept imaginable. That they ca…

[deleted]

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#19

Post by Zambyte »

abernard1 wrote: post #11
GeneralMayhem wrote: post #10

Earlier quoted context omitted.

The argument would be that that conceptual model is encoded in the intermediate-layer parameters of the model, in a different but analogous way to how it's encoded in the graph and chemical structure of your neurons.

I agree that's an argument. I would contend that argument is obviously false. If it were true, LLMs could multiply scalar numbers together trivially. It should be the easiest thing in the world for them. The network required to do that well is extremely small, the parameter sizes of these models are gigantic, and the textual expression is highly regular: multiplication is the simplest concept imaginable. That they ca…

> If it were true, LLMs could multiply scalar numbers together trivially.

FWIW most large models can do it better than I can in my head.

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#20

Post by LetsGetTechnicl »

jdietrich wrote: post #17
abernard1 wrote: post #11

Earlier quoted context omitted.

I agree that's an argument. I would contend that argument is obviously false. If it were true, LLMs could multiply scalar numbers together trivially. It should be the easiest thing in the world for them. The network required to do that well is extremely small, the parameter sizes of these models are gigantic, and the textual expression is highly regular: multiplication is the simplest concept imaginable. That they ca…

LLMs do contain conceptual representations and LLMs are capable of abstract reasoning. This is trivially provable by asking them to reason about something that is a) purely abstract and b) not in the training data, e.g. "All floots are gronks. Some gronks are klorps. Are any floots klorps?" Any of the leading LLMs will correctly answer questions of this type much more often than chance.

That is not an example of a LLM being capable of abstract reasoning. Changing the question from "What is the capital of United States?" which is easily answerable to something completely abstract and "not in the training model" doesn't change that LLM's are just very advanced text prediction, and always will be. The nature of their design means they are incapable of AGI.
Post reply on HN