Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Eight things to know about large language models [pdf]

cims.nyu.edu

Post reply on HN
Print view View original

11–20 of 114 posts

Re: Eight things to know about large language models [pdf]

#11

Post by cjbprime »

joe_the_user wrote: post #5

While I don't think these claims are entirely correct, I think they are worth considering. On the one hand, Gaining capabilities unpredictably is a bit exaggerated - it's more that a lot of apparent capabilities are embedded in language and these models approximate the truly vast amounts of text they've digested (imo). Just as much, LLMs don't express their creators values 'cause they don't express any values, they a…

> Gaining capabilities unpredictably is a bit exaggerated - it's more that a lot of apparent capabilities are embedded in language and these models approximate the truly vast amounts of text they've digested (imo)

I think the author would say those are not the capabilities being discussed here.

Re: Eight things to know about large language models [pdf]

#12

Post by cjbprime »

DonaldPShimoda wrote: post #3

This is a personal correspondence typeset via LaTeX — it is not an academic paper, and it was not peer-reviewed. (The document does not claim otherwise, but I think it's common for people to assume that documents that have been typeset in such a format are more rigorous than this is.) Leaving that aside, I really take issue with the style used by the author. For example, section 3 begins: > There is increasingly subs…

> LLMs do not "reason"; they do not "learn" or "develop" anything of their own volition.

If this claim of yours was sustainable, your grievance with the writing style would make a lot of sense. But GPT-4 can clearly reason about novel problems which were not in its training data. I think you should consider reading some of the many citations in the document which describe examples of that happening.

Re: Eight things to know about large language models [pdf]

#13

Post by brokenodo »

joe_the_user wrote: post #5

While I don't think these claims are entirely correct, I think they are worth considering. On the one hand, Gaining capabilities unpredictably is a bit exaggerated - it's more that a lot of apparent capabilities are embedded in language and these models approximate the truly vast amounts of text they've digested (imo). Just as much, LLMs don't express their creators values 'cause they don't express any values, they a…

Man, I have a totally opposite view about LLMs expressing creator’s values. Not only do they express them, they don’t STOP expressing them to the point of utter annoyance. Any remotely PG topic ends with a safety caveat, e.g., “however, it’s important to consider . . . .”

Re: Eight things to know about large language models [pdf]

#14

Post by judge2020 »

canjobear wrote: post #10
joe_the_user wrote: post #5

While I don't think these claims are entirely correct, I think they are worth considering. On the one hand, Gaining capabilities unpredictably is a bit exaggerated - it's more that a lot of apparent capabilities are embedded in language and these models approximate the truly vast amounts of text they've digested (imo). Just as much, LLMs don't express their creators values 'cause they don't express any values, they a…

> Gaining capabilities unpredictably is a bit exaggerated - it's more that a lot of apparent capabilities are embedded in language and these models approximate the truly vast amounts of text they've digested (imo). The fact is that they acquired abilities no one expected them to acquire. In hindsight you can say it was embedded in language and maybe you could have seen it coming, but it is an empirical fact that this…

It's still fairly exaggerated, because the LLM's purpose is basically "knowing human communication" and it still only exhibits that, but it knows how humans would respond if they obediently followed directions given to it exactly as instructed.

And it was only unexpected to those that weren't following the news the GPT 3 paper was published in 2020 and detailed this wild advancement in its capabilities[0,1].

0: https://www.infoq.com/news/2020/06/openai-gpt3-language-mode...

1: https://news.ycombinator.com/item?id=23885684

Re: Eight things to know about large language models [pdf]

#15

Post by DonaldPShimoda »

famouswaffles wrote: post #8
kenjackson wrote: post #4

Earlier quoted context omitted.

What does “reason” mean? It seems like it does everything I expect from something that reasons.

People routinely make up their own vague and ill defined meanings of understanding and reasoning to disqualify LLMs. This is necessary because LLMs obviously reason and understand by any evaluation that can be carried out. Seriously just watch. He's not actually going to be able to coherently define his "reasoning" in a way that can be tested.

> Seriously just watch. He's not actually going to be able to coherently define his "reasoning" in a way that can be tested.

Google gives the following definition of the verb "reason":

> think, understand, and form judgments by a process of logic.

LLMs do not think, they do not understand, and they do not form judgments. They do not come to their own conclusions. They do not have the physical capability. They are statistical models, nothing more.

> LLMs obviously reason and understand by any evaluation that can be carried out.

Uh-huh. Sure, Jan.

Re: Eight things to know about large language models [pdf]

#16

Post by SequoiaHope »

brokenodo wrote: post #13
joe_the_user wrote: post #5

While I don't think these claims are entirely correct, I think they are worth considering. On the one hand, Gaining capabilities unpredictably is a bit exaggerated - it's more that a lot of apparent capabilities are embedded in language and these models approximate the truly vast amounts of text they've digested (imo). Just as much, LLMs don't express their creators values 'cause they don't express any values, they a…

Man, I have a totally opposite view about LLMs expressing creator’s values. Not only do they express them, they don’t STOP expressing them to the point of utter annoyance. Any remotely PG topic ends with a safety caveat, e.g., “however, it’s important to consider . . . .”

Those statements don't come from the base model, they come from the steering methods, basically a form of hand tuning after the model is mostly trained, which the paper says are relatively crude and imperfect. It is the fact that these models are so unpredictable that led to these attempts at steering.

Re: Eight things to know about large language models [pdf]

#17

Post by TaylorAlexander »

DonaldPShimoda wrote: post #3

This is a personal correspondence typeset via LaTeX — it is not an academic paper, and it was not peer-reviewed. (The document does not claim otherwise, but I think it's common for people to assume that documents that have been typeset in such a format are more rigorous than this is.) Leaving that aside, I really take issue with the style used by the author. For example, section 3 begins: > There is increasingly subs…

> I hope the author can clarify their choice of phrasing in their work

From TFA's conclusions section:

"Open debates over whether we describe LLMs as understanding language, and whether to describe their actions using agency-related words like know or try, are largely separate from the questions that I discuss here (Bender & Koller, 2020; Michael, 2020; Potts, 2020). We can evaluate whether systems are effective or ineffective, reliable or unreliable, interpretable or uninterpretable, and improving quickly or slowly, regardless of whether they are underlyingly humanlike in the sense that these words evoke."

Re: Eight things to know about large language models [pdf]

#19

Post by famouswaffles »

DonaldPShimoda wrote: post #15
famouswaffles wrote: post #8

Earlier quoted context omitted.

People routinely make up their own vague and ill defined meanings of understanding and reasoning to disqualify LLMs. This is necessary because LLMs obviously reason and understand by any evaluation that can be carried out. Seriously just watch. He's not actually going to be able to coherently define his "reasoning" in a way that can be tested.

> Seriously just watch. He's not actually going to be able to coherently define his "reasoning" in a way that can be tested. Google gives the following definition of the verb "reason": > think, understand, and form judgments by a process of logic. LLMs do not think, they do not understand, and they do not form judgments. They do not come to their own conclusions. They do not have the physical capability. They are sta…

>LLMs do not think, they do not understand, and they do not form judgments. They do not come to their own conclusions. They do not have the physical capability. They are statistical models, nothing more.

"LLMs don't reason because they don't understand" is not the bastion of genius you think it is. It's a circular argument that relies on whatever bespoke interpretations you have cooked up.

They don't form judgement or conclusions? Sure looks like they do. So what's the difference ?

What is GPT-4 doing then when it correctly looks like it is reasoning and what's the difference between that and "real" understanding or reasoning.

Such a huge difference I should be able to test for it. Don't understand how you can tell me what I'm seeing isn't real reasoning but fail to provide a way to empirically determine the difference.

>Uh-huh. Sure, Jan.

Yeah. https://arxiv.org/abs/2212.09196 Many other evaluations to carry out.

Re: Eight things to know about large language models [pdf]

#20

Post by kqr »

DonaldPShimoda wrote: post #15
famouswaffles wrote: post #8

Earlier quoted context omitted.

People routinely make up their own vague and ill defined meanings of understanding and reasoning to disqualify LLMs. This is necessary because LLMs obviously reason and understand by any evaluation that can be carried out. Seriously just watch. He's not actually going to be able to coherently define his "reasoning" in a way that can be tested.

> Seriously just watch. He's not actually going to be able to coherently define his "reasoning" in a way that can be tested. Google gives the following definition of the verb "reason": > think, understand, and form judgments by a process of logic. LLMs do not think, they do not understand, and they do not form judgments. They do not come to their own conclusions. They do not have the physical capability. They are sta…

This is the problem with non-operational definitions, because now we need to know how you define "think" and "understand" and "form judgments", to move on.

Instead, could you operationally define "reason" in a way that a human is, say, 90 % likely to pass the test and GPT is 10 % likely to do?

Post reply on HN