LLMs don't do formal reasoning

Tool use is the measure of intelligence. Terence Tao can use this tool for mathematics.

When Google came out, search engines were suddenly more useful. But there were a bunch of people talking about how “Not everything they find is right” and how “that is a huge problem”.

Then for two decades, people used search highly successfully. Fascinating thing. Tool use.

At one point the goal posts was the Turing test. That’s long since been passed, and we aren’t satisfied.

Then goal posts were moved to logical reasoning such as the Winograd Schemas. Then that wasn’t enough.

In fact, it’s abundantly clear we won’t be satisfied until we’ve completely destroyed human intelligence as superior.

The current goal post is LLMs must do everything better than humans or it’s not AGI. If there is one thing it does worse, people will cite it as just a stochastic parrot. That’s a complete fallacy.

Of course we dare not compare LLMs to the worse case human - because LLMs would be AGI compared to that.

We compare LLMs to the best human in every category - unfairly.

With LLMs it’s been abundantly clear - there is not a line where something is intelligent or not. There’s only shades of gray and eventually we call it black.

There will always be differences between LLM capabilities and humans - different architectures and different training. However it’s very clear that a process that takes huge amounts of data and processes it whether a brain or LLM come up with similar results.

Someone should up with a definition of intelligence that excludes all LLMs and includes all humans.

Also while you are at it, disprove humans do more than what ChatGPT does - aka probabilistic word generation.

I’ll wait.

Until then, as ChatGPT blows past what was science fiction 5 years ago, maybe these arguments aren’t great?

Also - name one thing we have the data for that we haven’t been able to produce a neural network capable of performing that task?

Human bodies have so many sensors it’s mind blowing. The data any human processes in one days simply blows LLMs out of the water.

Touch, taste, smell, hearing, etc…

That’s not to say if you could hook up a hypothetical neural network to a human body, that we couldn’t do the same.

Is hackernews..hacked?

He is coming from the perspective of a long-running debate on symbolic versus statistical/data-driven approaches to modeling language structure and use. It seems in recent years he has had trouble coming to terms with the fact that at least for real-world applications of language technology, the statistical approach has simply won the war (or at worst, forms the core foundation on top of which symbolic approaches can have some utility).

I come from the same academic tradition, and have colleagues in common with him. He has been advocating for a quasi-chomskyan perspective on language science for many years -- as have many others working at the intersection of linguistics and psychology/cog sci.

TBH I suspect he himself is a large part of his target audience. A lot of older school academics raised in the symbolic tradition are pretty unsettled by the incredible achievements of the data-driven approach.

Personally I saw the writing on the wall years ago and have transitioned to working in statistical NLP (or "AI" I suppose). Feeling pretty good about that decision these days.

FWIW I do think symbolic approaches will start to shine in the next several years, as a way to control the behavior of modern statistical LMs. But doubtful they will ever produce anything comparable to current systems without a strong base model trained on troves of data.

edit: Worth noting that Marcus has produced plenty of high-quality research in his career. I think his main problem here is that he seems to believe that AI systems should function analogously to how human language/cognition functions. But from an engineering/product perspective, how a system works is just not that important compared to how well it works. There's probably a performance ceiling for purely statistical models, and it seems likely that some form of symbolic machinery can raise that ceiling a bit. Techniques that work will eventually make their way into products, no matter which intellectual tradition they come from. But framing things in this way is just not his style.