Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Eight things to know about large language models [pdf]

cims.nyu.edu

Post reply on HN
Print view View original

31–40 of 114 posts

Re: Eight things to know about large language models [pdf]

#31

Post by avereveard »

DonaldPShimoda wrote: post #3

This is a personal correspondence typeset via LaTeX — it is not an academic paper, and it was not peer-reviewed. (The document does not claim otherwise, but I think it's common for people to assume that documents that have been typeset in such a format are more rigorous than this is.) Leaving that aside, I really take issue with the style used by the author. For example, section 3 begins: > There is increasingly subs…

While llm don't learn, they have now enough token input space and handle context well enough to manage a cyclic conversation.

While not learning, they can still run inferences on what they are fed as context within the token space. And because they are becoming so good at getting context out of structured data, they can use the context provided as a layer on top of their knowledge.

So you can at a very good approximation upload knowledge into them, and such knowledge can represent how to use tools to fetch data they don't know about - i.e. what langchain agents are doing.

Now of course that is not 'llm' some, is llm + code for informaron discovery, but that code is entirely supported and enabled by the existence of good enough llm

Re: Eight things to know about large language models [pdf]

#32

Post by graypegg »

kenjackson wrote: post #27
graypegg wrote: post #9

Earlier quoted context omitted.

I’ve been imagining “reasoning” as meaning “referring to an abstraction”. The idea being if you wanted to write a script to tell you how far an arrow of a given weight will fly, you’ll probably use some physics simulation (or even just a coefficient to multiply the input weight by) to abstract the arrow in flight. Then you can answer questions around that abstraction. The alternative is having 1000 different results…

Does it really provide the statistically most common next token? I don’t think that’s even true. And we’ve seen what looks like emergent behavior as parameters increase. I suspect it is actually creating abstractions at a deeper level than standard DNNs.

I’d be curious to know! I get skeptical when it starts moving into the “emergent properties of language” but language was built by fleshy neurons, it could be that some inherent pattern emerges in the language we speak that reflects the inner workings of our brain, which by trying to emulate makes a similar brain on the other end.

However, that feels a bit magical to me at the moment. Still interesting to be along for the ride humanity is going on!

Re: Eight things to know about large language models [pdf]

#33

Post by sanxiyn »

PaulDavisThe1st wrote: post #26
sanxiyn wrote: post #22

Earlier quoted context omitted.

LLMs clearly develop internal representations, this is an empirical fact. For example, one paper cited is https://arxiv.org/abs/2210.13382 , which trained an LLM on Othello games, where an 8 layer model is fed one of 60 tokens (there are 60 possible moves in Othello, 8*8 minus 4 initially occupied squares). You can probe this model at layer 5, and it computes Othello board state from moves, and you can be sure this b…

> LLMs clearly develop internal representations, this is an empirical fact. you've offered an anecdote. Some LLMs (more generally, this type of neural network model) will generate configurations that can be understood as a representation; others will not. The fact that the authors were able to find an apparent model in a heavily rule-based system is not incredibly surprising but offers little clue about whether this…

Okay, I think we are in agreement. Let's say current LLM architectures are clearly capable of developing internal representations, not just learning surface statistics, and it can execute algorithmic computation as complex as computing Othello board states, and it can completely generalize out of distribution thanks to such algorithmic computation. (One experiment was to completely eliminate any Othello games starting with move C5 from the training data. The model had no trouble generating only legal moves for C5 games.)

I agree whether LLMs trained on natural language do develop world model is an interesting and open empirical question. They clearly can, but we don't know whether they do. I do consider the evidence pretty suggestive that they do. But this is clearly just an empirical question, not a question that can be answered by arguments as useless as "they are statistical models", because statistical models can and do learn world model.

I actually consider your considering this result not incredibly surprising pretty surprising.

Re: Eight things to know about large language models [pdf]

#34

Post by UncleEntity »

DonaldPShimoda wrote: post #3

This is a personal correspondence typeset via LaTeX — it is not an academic paper, and it was not peer-reviewed. (The document does not claim otherwise, but I think it's common for people to assume that documents that have been typeset in such a format are more rigorous than this is.) Leaving that aside, I really take issue with the style used by the author. For example, section 3 begins: > There is increasingly subs…

So the AI Ethics peeps are the True Scotsmen?

We may just live in a world where there’s room for more than one viewpoint and people are able to research different aspects of the same thing without getting into debates over dogma.

I mean, inclusion -> https://news.ycombinator.com/item?id=34698769

Re: Eight things to know about large language models [pdf]

#35

Post by jawiggins »

> 6. Human performance on a task isn’t an upper bound on LLM performance

Is that true? Because LLMs are trained on the work of human: a LLM that learned all of the content of the articles would at best yield output equal to that of the human writing.

The reasons the author gives seem unfounded to me:

> First, they are trained on far more data than any human sees

If the human writer has access to google (everyone in modern society) this point is moot.

> In addition, they are often given additional training using reinforcement learning before being deployed.

If this is human in the loop RL, then the upper bound would still be the human training it. If it isn't, refer to #1.

Re: Eight things to know about large language models [pdf]

#36

Post by cjbprime »

graypegg wrote: post #29
cjbprime wrote: post #23

Earlier quoted context omitted.

This is one of those comments that begs for actually using LLMs. Yes, an advanced LLM like GPT-4 will absolutely show evidence of an abstracted world model, correctly answering questions about where person X thinks object Y is, or what relation two people have to each other, or playing a text adventure game while clearly following an internal map it has created for itself.

So I get that the evidence is that you can do insanely complex things with consistency, like playing a text adventure. You pick up a thing and it stays in your inventory etc, ergo, it’s modelled a character with an inventory in a text adventure. vvv But, coming at this as *respectfully and curiously* as I can here: ^^^ I guess I’m a little more skeptical about attributing magic to something that I know is only workin…

> something that I know is only working from words as a source.

Your intuition about the architecture might be causing the distrust. Yes, the ends of the model are words in and words out (although GPT-4 is now multi-modal and can accept images as input too), but that doesn't mean the calculations in the middle are dealing in words. The "language" in "large language model" is misleading. It's a large generalized model that imports and exports language.

Re: Eight things to know about large language models [pdf]

#37

Post by ttpphd »

cjbprime wrote: post #12
DonaldPShimoda wrote: post #3

This is a personal correspondence typeset via LaTeX — it is not an academic paper, and it was not peer-reviewed. (The document does not claim otherwise, but I think it's common for people to assume that documents that have been typeset in such a format are more rigorous than this is.) Leaving that aside, I really take issue with the style used by the author. For example, section 3 begins: > There is increasingly subs…

> LLMs do not "reason"; they do not "learn" or "develop" anything of their own volition. If this claim of yours was sustainable, your grievance with the writing style would make a lot of sense. But GPT-4 can clearly reason about novel problems which were not in its training data. I think you should consider reading some of the many citations in the document which describe examples of that happening.

It doesn't reason, it produces output that better conforms to your expectations of a reasoning actor.

Re: Eight things to know about large language models [pdf]

#38

Post by ttpphd »

kenjackson wrote: post #4
DonaldPShimoda wrote: post #3

This is a personal correspondence typeset via LaTeX — it is not an academic paper, and it was not peer-reviewed. (The document does not claim otherwise, but I think it's common for people to assume that documents that have been typeset in such a format are more rigorous than this is.) Leaving that aside, I really take issue with the style used by the author. For example, section 3 begins: > There is increasingly subs…

What does “reason” mean? It seems like it does everything I expect from something that reasons.

When you hear someone speak, are you thinking their thoughts?

We are talking about a model of language and our expectations for what the words would look like if a person were using language to reason with. People are confusing one thing (their own interpretation of what they are reading) with another (a statistical model that is being driven to align with certain specific human expectations).

Re: Eight things to know about large language models [pdf]

#39

Post by jcparkyn »

jawiggins wrote: post #35

> 6. Human performance on a task isn’t an upper bound on LLM performance Is that true? Because LLMs are trained on the work of human: a LLM that learned all of the content of the articles would at best yield output equal to that of the human writing. The reasons the author gives seem unfounded to me: > First, they are trained on far more data than any human sees If the human writer has access to google (everyone in m…

> If the human writer has access to google (everyone in modern society) this point is moot.

But a single human can only read so much information in a finite amount of time (their life). Sure, the same thing applies to LLMs, but the theoretical limits are orders of magnitude higher.

And while this probably isn't what you were talking about, it's also pretty clear that any time-limited task favours LLMs. There's no human on earth that could write a non-trivial short story in 10 seconds, regardless of quality level.

Re: Eight things to know about large language models [pdf]

#40

Post by cjbprime »

Godel_unicode wrote: post #28
cjbprime wrote: post #24

Earlier quoted context omitted.

New bar for people claiming LLMs can't reason: invent a specific, testable problem, representable in text, that many humans can solve and LLMs can't, and tell us what it is.

… and then perform a careful search of books and the whole internet to be sure what you think is novel hasn’t been thoroughly debated somewhere on stackexchange.

If it's a stochastic parrot, then merely randomizing proper nouns and filler text should be enough to prevent its abstraction ability.

If you're saying that we can't use a problem if any analog of that problem has ever been described, you seem to be arguing more strongly that it is a general intelligence than I am.

Post reply on HN