Eight things to know about large language models [pdf]

Post by somenameforme » Tue, Apr 04, 2023, 7:44 AM UTC

SanderNL wrote: post #70

somenameforme wrote: post #57

Earlier quoted context omitted.

It's a word prediction algorithm. Literally any collection of words (sometimes also known as a fact) it was trained on, regardless of how esoteric or domain specific, should generally be able to be regurgitated and, to a lesser degree, associated with similar collections of words. If you want to see it fail, don't try to "outsmart" it, but simply consider how it's programmed. --- Me: "continue the sequence: 0, done,…

I apologize for being enormously stupid, but the word is a rhyme with the underlying number right? Edit: I want to add I even get GPT3 getting it right without explanation to be honest. Even that one is outsmarting me. 4 is using terms like homophone. I'm not sure you proved what you thought you proved.

Post by MattPalmer1086 » Tue, Apr 04, 2023, 7:47 AM UTC

OmegaPoint wrote: post #67

somenameforme wrote: post #57

Earlier quoted context omitted.

It's a word prediction algorithm. Literally any collection of words (sometimes also known as a fact) it was trained on, regardless of how esoteric or domain specific, should generally be able to be regurgitated and, to a lesser degree, associated with similar collections of words. If you want to see it fail, don't try to "outsmart" it, but simply consider how it's programmed. --- Me: "continue the sequence: 0, done,…

I asked my coworkers in the office but none was able to answer that. Not sure because they were are ESL (me included) or because they were GPT in disguise.

Post by kqr » Tue, Apr 04, 2023, 7:54 AM UTC

sanxiyn wrote: post #47

kqr wrote: post #20

Earlier quoted context omitted.

This is the problem with non-operational definitions, because now we need to know how you define "think" and "understand" and "form judgments", to move on. Instead, could you operationally define "reason" in a way that a human is, say, 90 % likely to pass the test and GPT is 10 % likely to do?

Yes, François Chollet released ARC(Abstraction and Reasoning Corpus) benchmark for this in 2019, and the benchmark can be scored automatically. Humans solve 100% of tests and GPTs solve 0% of tests and GPTs made exactly zero progress from 2019 to 2022. https://twitter.com/fchollet/status/1631699463524986880 https://github.com/fchollet/ARC

Post by SanderNL » Tue, Apr 04, 2023, 8:11 AM UTC

ttpphd wrote: post #37

cjbprime wrote: post #12

Earlier quoted context omitted.

> LLMs do not "reason"; they do not "learn" or "develop" anything of their own volition. If this claim of yours was sustainable, your grievance with the writing style would make a lot of sense. But GPT-4 can clearly reason about novel problems which were not in its training data. I think you should consider reading some of the many citations in the document which describe examples of that happening.

It doesn't reason, it produces output that better conforms to your expectations of a reasoning actor.

Post by somenameforme » Tue, Apr 04, 2023, 8:21 AM UTC

SanderNL wrote: post #72

somenameforme wrote: post #68

Earlier quoted context omitted.

It's not about directly regurgitating the training set, but about creating associations that allow it to indirectly regurgitate, and more critically - mix, its training set. It's easier to understand this in a different domain. Imagine we want to make a letter parsing algorithm. The first step is to simplify the input and create a consistent format. So we might, for instance, convert all inputs to a 100x100 black/whi…

You just explained how abstraction and intelligence in general works. > but can consistently replicate not only its training but even more beyond that We call that generalization. No?