Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4

arxiv.org

Post reply on HN
Print view View original

21–30 of 62 posts

Re: Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4

#21

Post by dmreedy »

svachalek wrote: post #19
Closi wrote: post #3

I think one main failure in the framing of these papers (and discussion of LLMs more broadly) is that the abstract says that GPT4 ‘struggles’ with logical reasoning: > ChatGPT and GPT-4 do relatively well on well-known datasets […] however, the performance drops significantly when handling newly released and out-of-distribution [where] Logical reasoning remains challenging for ChatGPT and GPT-4 But reading the paper…

If you really force it to reason, rather than regurgitate arguments from its training set, you will find it is nowhere near the genius line. Make up some rules and have it try to answer questions according to the rules. In my experiments I feel it's something like a 4 or 5 year old child both in its logical limitations and penchant for distraction. However it's important to note one VERY important thing -- this is no…

> this is not a system that is designed to reason

For what it's worth, neither are we, really. Not disagreeing with anything you're saying, just musing.

> superhuman levels of reasoning

This one has always stumped me a bit though. I'm not quite sure what that looks like. Laplace's Demon?

Re: Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4

#22

Post by raydiatian »

outofpaper wrote: post #14
raydiatian wrote: post #8

Earlier quoted context omitted.

I have been thinking for a few weeks now that we need another term for large language models trained on colossal datasets: AGK, artificially generally/globally knowledgeable. It can mimic a likeness of problem solving because the corpus it was trained on is full of problem/solution pairs in the abstract. But task it with any novel problem solving challenge outside of its training that is of sufficient complexity and…

Often it can actually solve more complex problems but needs to have its "hand held". Essentially the model needs to be guided to/through problem solving techniques. We have to remember that LLM are literally inference engines. They default to providing us with probable results, probable responses. They can pe pulled away from these "knee jerk" responses.

Right but if it needs its hand held, that ends up being a transcription task rather than a logical reasoning task. Like if you _tell it_ the solution to a coding job in detail, it can build you the complex entity you’re looking for. But if you just say for instance “write me a Python script that generates random chord changes (ex A#dim to Gmaj9b5)”, first of all it will just dump code without asking for clarification on requirements, and second of all even if you do give it further clarification on requirements the code won’t work without you explaining in depth the algorithm.

Although, that’s just a personal anecdote.

Re: Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4

#24

Post by dmreedy »

mftb wrote: post #17
pottspotts wrote: post #7

Earlier quoted context omitted.

For a significant number of software developers, GPT and Github's Copilot have replaced StackOverflow, and even Googling more generally. It is more than an autocomplete, it is the best resource for software development by far, IMO. It's a tutor that's an expert in virtually every topic.

Yea it's not. Sorry to contradict, but it's not like that. In any kind of tutoring arrangement you're time with them is limited, and if they're any good, they don't just regurgitate limitless example code. Two of the most important decisions that an instructor has to make are, how much access to give you, and how much example material to give you, because the actual learning begins when you have to think for yourself…

Tutoring assumes the skill is valuable to learn, that there is a need for more people who know how to do it.

We don't really tutor people how to write too much assembly anymore, or hand-compile code. So if you're arguing that ChatGPT meets the definition of a tool, or a servant, better than a tutor, fair, but if you're further arguing that that makes it somehow less valuable than a tutor (in this case), I'm not sure I can come along there.

Re: Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4

#25

Post by dragonwriter »

outofpaper wrote: post #14
raydiatian wrote: post #8

Earlier quoted context omitted.

I have been thinking for a few weeks now that we need another term for large language models trained on colossal datasets: AGK, artificially generally/globally knowledgeable. It can mimic a likeness of problem solving because the corpus it was trained on is full of problem/solution pairs in the abstract. But task it with any novel problem solving challenge outside of its training that is of sufficient complexity and…

Often it can actually solve more complex problems but needs to have its "hand held". Essentially the model needs to be guided to/through problem solving techniques. We have to remember that LLM are literally inference engines. They default to providing us with probable results, probable responses. They can pe pulled away from these "knee jerk" responses.

> Often it can actually solve more complex problems but needs to have its “hand held”. Essentially the model needs to be guided to/through problem solving techniques.

While I haven’t done experiments with it hooked up to enough resources to really solve problems autonomously, providing it access to lookup information (e.g., searching wikipedia) and do simply computation (e.g., send python expressions to be evaluated) it figures out a lot more than just the chat interface alone without resources, without hand holding. I think autonomously solving problems where the necessary information is in the universe covered by training data and accessible resources is not unrealistic.

Re: Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4

#26

Post by stephendause »

svachalek wrote: post #19
Closi wrote: post #3

I think one main failure in the framing of these papers (and discussion of LLMs more broadly) is that the abstract says that GPT4 ‘struggles’ with logical reasoning: > ChatGPT and GPT-4 do relatively well on well-known datasets […] however, the performance drops significantly when handling newly released and out-of-distribution [where] Logical reasoning remains challenging for ChatGPT and GPT-4 But reading the paper…

If you really force it to reason, rather than regurgitate arguments from its training set, you will find it is nowhere near the genius line. Make up some rules and have it try to answer questions according to the rules. In my experiments I feel it's something like a 4 or 5 year old child both in its logical limitations and penchant for distraction. However it's important to note one VERY important thing -- this is no…

Re: your last sentence, I'm fairly certain this would fall under the category of neuro-symbolic AI. It, too, seems to me like the logical next step.

https://en.m.wikipedia.org/wiki/Neuro-symbolic_AI

Re: Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4

#27

Post by raydiatian »

famouswaffles wrote: post #16
raydiatian wrote: post #8

Earlier quoted context omitted.

I have been thinking for a few weeks now that we need another term for large language models trained on colossal datasets: AGK, artificially generally/globally knowledgeable. It can mimic a likeness of problem solving because the corpus it was trained on is full of problem/solution pairs in the abstract. But task it with any novel problem solving challenge outside of its training that is of sufficient complexity and…

Reading the paper it seems they are problems a lot of people would fail at it too, at least some of the time. LLMs are not superhuman in logical reasoning seems to be the conclusion more than anything.

What you’re saying gets to the core of why I would call it AGK and not AGI. Training a transformer on known answers to problems and then observing that it can successfully answer questions related to those problems is cheating.

I think the way that Ilya suggests that the “test for consciousness is to train a model with an absolute absence of any training example remotely referring to the notion of a self or of feeling, and then ask it questions about feeling. If the model can do it, congrats, you’ve discovered consciousness.” Similarly, if you train an architecture on exclusively the building blocks of a particular class of problem, and also avoid training it on any sort of problem where it could just reason by analogy and get a correct answer (isolating first principles thing as the only option), then if it can solve the problem you have a genuine problem solving architecture.

Re: Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4

#29

Post by famouswaffles »

raydiatian wrote: post #27
famouswaffles wrote: post #16

Earlier quoted context omitted.

Reading the paper it seems they are problems a lot of people would fail at it too, at least some of the time. LLMs are not superhuman in logical reasoning seems to be the conclusion more than anything.

What you’re saying gets to the core of why I would call it AGK and not AGI. Training a transformer on known answers to problems and then observing that it can successfully answer questions related to those problems is cheating. I think the way that Ilya suggests that the “test for consciousness is to train a model with an absolute absence of any training example remotely referring to the notion of a self or of feelin…

Meh Intelligence is Intelligence.

It's not cheating for people so asserting that it's cheating for machines just seems like goal post shifting more than anything.

Like this idea to pass the machines through frankly ridiculous hoops that humans wouldn't even pass is just..ehh. you seen how children with no language development in childhood turn out ?

It just misses the point entirely.

It's like the user down the thread said. Some isolate groups will build asi while the rest of the world is bickering about philosophical zombies and consciousness.

Re: Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4

#30

Post by inopinatus »

pottspotts wrote: post #7
RayVR wrote: post #2

This aligns well with my personal experience using gpt-4. The model provides surprisingly good responses on topics which I know are readily available online while being potentially troublesome to find the exact information I want. I have even found it useful when I know there is a tool for what I want but can’t recall the jargon used to find it via Google. Simply describing the rough idea is enough to get the model t…

For a significant number of software developers, GPT and Github's Copilot have replaced StackOverflow, and even Googling more generally. It is more than an autocomplete, it is the best resource for software development by far, IMO. It's a tutor that's an expert in virtually every topic.

It really isn't. GPT-4 is certainly an improvement over previous language models, but when I vaingloriously gave it the questions from favourite self-answers on StackOverflow, only one completion was immediately correct. The remainder were variously suboptimal, poorly crafted, overdesigned, incomplete, or downright wrong, requiring multiple re-prompts to coax into usable condition. The they were all syntactically valid but tended to misconstrue the semantics and underestimate the capabilities of the programming environments concerned. Try it with your own, but to me it's more like coaching a bright but inexperienced junior developer with the "confidently incorrect" trait.
Post reply on HN