LLMs don't do formal reasoning
garymarcus.substack.com
LLMs don't do formal reasoning
1–10 of 128 posts
Re: LLMs don't do formal reasoning
#2If anyone is curious, a Meta Data Scientist published a great piece about how the facts about what LLMs are actually doing (and therefore able to do) and how it's papered over by using chat bots. It's a long but very engaging read.
https://medium.com/@colin.fraser/who-are-we-talking-to-when-...
Re: LLMs don't do formal reasoning
#3Well, it's about time we moved the goalposts again. All this business about trophies fitting in suitcases being the gold standard was getting pretty embarrassing.
Re: LLMs don't do formal reasoning
#4You could substitute "LLMs" -> "Humans" and the statement would also be true.
Re: LLMs don't do formal reasoning
#5This trope of proclaiming some critical flaw in the functioning of LLMs with the implication that they therefore should not be used is getting boring.
LLMs are far from perfect but they can be a very useful tool that, used well, can add significant value in spite of their flaws. Large numbers of people and businesses are extracting huge value from the use of LLMs every single day. Some people are building what will become wildly successful businesses around LLM technology.
Yet in the face of this we still see a population of naysayers who appear intent on rubbishing LLMs at any cost. To me that seems like a pretty bad faith dialogue.
I’m aware that a lot of the positive rhetoric, particularly early on after the first public release of ChatGPT was overstated - sometimes heavily so - but taking one set of shitty arguments and rhetoric and responding to it with polar opposite, but equally shitty, arguments and rhetoric for the most part only serves to double the quantity of shitty arguments and rhetoric (and, adding insult to injury, often does so in the name of “balance”).
Re: LLMs don't do formal reasoning
#6You could substitute "LLMs" -> "Humans" and the statement would also be true.
the article describes pretty specific failures that humans of even below-average intelligence do not make
Re: LLMs don't do formal reasoning
#7You could substitute "LLMs" -> "Humans" and the statement would also be true.
Are you suggesting Humans can't do formal reasoning? Because you can easily teach a four year old not to make illegal moves in chess with very little instructions, and by 10 geniuses like Terence Tao were discussing open math problems with Erdos. If anything this article adds further evidence that whatever the architecture of the human brain it is very different to an LLM architecture.
Re: LLMs don't do formal reasoning
#8ChatGPT-4o:
44+58+88=190
So, Oliver has a total of 190 kiwis. The five smaller kiwis on Sunday are still included in the total count, so they don't change the final sum.
Re: LLMs don't do formal reasoning
#9Integer multiplication was used to test LLMs reasoning capabilities, and I think Karpathy mentioned that tokenization might play a role in basic math. MathGLM was compared against GPT-4 in the article, but I couldn't figure out if MathGLM was trained with character-level tokenization or not.
Re: LLMs don't do formal reasoning
#10They’re arguing since it’s not close to perfect, it’s not useful? Seems like a straw man.