Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language is not essential for the cognitive processes that underlie thought

scientificamerican.com

Post reply on HN
Print view View original

131–140 of 424 posts

Re: Language is not essential for the cognitive processes that underlie thought

#131

Post by kridsdale1 »

HarHarVeryFunny wrote: post #54
Animats wrote: post #48

This is an important result. The actual paper [1] says that functional MRI (which is measuring which parts of the brain are active by sensing blood flow) indicates that different brain hardware is used for non-language and language functions. This has been suspected for years, but now there's an experimental result. What this tells us for AI is that we need something else besides LLMs. It's not clear what that someth…

Brain size isn't necessarily a very good correlate of intelligence. For example dolphins and elephants have bigger brains than humans, and sperm whales have much bigger brains (5x by volume). Neanderthals also had bigger brains than modern humans, but are not thought to have been more intelligent. A crow has a small brain, but also has very small neurons, so ends up having 1.5B neurons, similar to a dog or some monke…

Don’t assume whales are less intelligent than humans. They’re tuned for their environment. They won’t assemble machines with their flippers but let’s toss you naked in the pacific and see if you can communicate and collaborate with peers 200km away on complex hunting strategies.

Re: Language is not essential for the cognitive processes that underlie thought

#132

Post by alphan0n »

fhdsgbbcaA wrote: post #118
theptip wrote: post #98

Earlier quoted context omitted.

LLM as a term is becoming quite broad; a multi-modal transformer-based model with function calling / ReAct finetuning still gets called an LLM, but this scaffolding might be all that’s needed. I’d be extremely surprised if AI recapitulates the same developmental path as humans did; evolution vs. next-token prediction on an existing corpus are completely different objective functions and loss landscapes.

I asked both OpenAI and Claude the same difficult programming question. Each gave a nearly identical response down to the variable names and example values. I then looked it up and they had each copy/pasted the same Stack overflow answer. Furthermore, the answer was extremely wrong, the language I used was superficially similar to the source material, but the programming concepts were entirely different. What this te…

What was the question?

Re: Language is not essential for the cognitive processes that underlie thought

#133

Post by KoolKat23 »

fuzzfactor wrote: post #78
Animats wrote: post #75

Earlier quoted context omitted.

I don't know what we need. Nor does anybody else, yet. But we know what it has to do . Basically what a small mammal or a corvid does. There's been progress. Look at this 2020 work on neural net controlled drone acrobatics.[1] That's going in the right direction. [1] https://rpg.ifi.uzh.ch/docs/RSS20_Kaufmann.pdf

You could say language is just the "communication module" but there has got to be another whole underlying interface where non-verbal thoughts are modulated/demodulated to conform to the language expected to be used when communication may or may not be on the agenda.

As far as I understand it, it's just output and speaking is just enclosed in tags, that the body can act on, much like inline code output from an LLM.

e.g. the neural electrochemical output has a specific sequence that triggers the production of a certain hormone in your pituitary gland for e.g. and the hormone travels to the relevant body function activating/stopping it.

Re: Language is not essential for the cognitive processes that underlie thought

#134

Post by vundercind »

fhdsgbbcaA wrote: post #118
theptip wrote: post #98

Earlier quoted context omitted.

LLM as a term is becoming quite broad; a multi-modal transformer-based model with function calling / ReAct finetuning still gets called an LLM, but this scaffolding might be all that’s needed. I’d be extremely surprised if AI recapitulates the same developmental path as humans did; evolution vs. next-token prediction on an existing corpus are completely different objective functions and loss landscapes.

I asked both OpenAI and Claude the same difficult programming question. Each gave a nearly identical response down to the variable names and example values. I then looked it up and they had each copy/pasted the same Stack overflow answer. Furthermore, the answer was extremely wrong, the language I used was superficially similar to the source material, but the programming concepts were entirely different. What this te…

They don’t wonder. They’d happily produce entire novels of (garbage) text if trained on gibberish. They wouldn’t be confused. They wouldn’t hope to puzzle out the meaning. There is none, and they work just fine anyway. Same for real language. There’s no meaning, to them (there’s not really a “to” either).

The most interesting thing about LLMs is probably how much relational information turns out to be encoded in large bodies of our writing, in ways that fancy statistical methods can access. LLMs aren’t thinking, or even in the same ballpark as thinking.

Re: Language is not essential for the cognitive processes that underlie thought

#135

Post by [deleted] »

KoolKat23 wrote: post #126
Animats wrote: post #75

Earlier quoted context omitted.

I don't know what we need. Nor does anybody else, yet. But we know what it has to do . Basically what a small mammal or a corvid does. There's been progress. Look at this 2020 work on neural net controlled drone acrobatics.[1] That's going in the right direction. [1] https://rpg.ifi.uzh.ch/docs/RSS20_Kaufmann.pdf

I think you may underestimate what these models do. Proper multimodal models natively consider whatever input you give them, store the useful information in an abstracted form (i.e not just text), building it's world model, and then output in whatever format you want it to. It's no different to a mammals, just the inputs are perhaps different. Instead of relying on senses, they rely on text, video, images and sound.…

[deleted]

Re: Language is not essential for the cognitive processes that underlie thought

#136

Post by Razengan »

pineaux wrote: post #117
datameta wrote: post #36

Earlier quoted context omitted.

I can remember the sequence of sounds and like a delay line repeat that sequence in my head. This becomes easier the more distinguishable the taps are or the more of a cadence variability there is. But if it is a longer sequence I compress it by remembering an analogue like so: doo doo da doo da doo da da doo (reminiscent of morse code, or a kind of auditory binary). Would we consider this language? I think in the co…

I think this is what language is. It's a sequence rememberance system.

Oh no… That would vindicate the chatbots..

Re: Language is not essential for the cognitive processes that underlie thought

#137

Post by Razengan »

fnordpiglet wrote: post #81
Razengan wrote: post #21

Earlier quoted context omitted.

Can you count without using a "language"? Try it now: Tap your hand on the desk randomly. Can you recall how many times you did it without "saying" a sequence in your head like "1, 2, 3" or "A, B, C" etc? If yes, how far can you count? With a language it's effectively infinite. You could theoretically go up to "1 million 5 hundred 43 thousand, 2 hundred and 10" and effortlessly know what comes next.

I don’t make a sound or word in my mind but I definitely keep track of the number. My thinking is definitely structured and there are things in my thoughts but there is no words or voice. I also can’t see images in my mind either. I’ve no idea what an inner monologue or the minds eye is like. I have however over the years found ways to produce these experiences in a way of my own. I found for instance some rough visu…

This is So unrelatable lol. Imagine how different alien minds would be!!

Re: Language is not essential for the cognitive processes that underlie thought

#139

Post by YeGoblynQueenne »

>> They’re basically the first model organism for researchers studying the neuroscience of language. They are not a biological organism, but until these models came about, we just didn’t have anything other than the human brain that does language.

I think this is completely wrong-headed. It's like saying that until cars came about we just didn't have anything other than animals that could move around under its own power, therefore in order to understand how animals move around we should go and study cars. There is a great gulf of unsubstantiated assumptions between observing the behaviour of a technological artifact, like a car or a statistical language model, and thinking we can learn something useful from it about human or more generally animal faculties.

I am really taken aback that this is a serious suggestion: study large language models as in-silico models of human linguistic ability. Just putting it down in writing like that rings alarm bells all over the place.

Re: Language is not essential for the cognitive processes that underlie thought

#140

Post by jebarker »

awongh wrote: post #120
jebarker wrote: post #93

Earlier quoted context omitted.

> What this tells us for AI is that we need something else besides LLMs Not to over-hype LLMs, but I don't see why this results says this. AI doesn't need to do things the same way as evolved intelligence has.

One reason might that LLMs are successful because of the architecture, but also, just as importantly because they can be trained over a volume and diversity of human thought that’s encapsulated in language (that is on the internet). Where are we going to find the equivalent data set that will train this other kind of thinking? Open AI O1 seems to be trained on mostly synthetic data, but it makes intuitive sense that…

I think the data is way more important for the success of LLMs than the architecture although I do think there's something important in the GPT architecture in particular. See this talk for why: [1]

Warning, watch out for waving hands: The way I see it is that cognition involves forming an abstract representation of the world and then reasoning about that representation. It seems obvious that non-human animals do this without language. So it seems likely that humans do too and then language is layered on top as a turbo boost. However, it also seems plausible that you could build an abstract representation of the world through studying a vast amount of human language and that'll be a good approximation of the real-world too and furthermore it seems possible that reasoning about that abstract representation can take place in the depths of the layers of a large transformer. So it's not clear to me that we're limited by the data we have or necessarily need a different type of data to build a general AI although that'll likely help build a better world model. It's also not clear that an LLM is incapable of the type of reasoning that animals apply to their abstract world representations.

[1] https://youtu.be/yBL7J0kgldU?si=38Jjw_dgxCxhiu7R

Post reply on HN