Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deciphering language processing in the human brain through LLM representations

research.google

Post reply on HN
Print view View original

71–80 of 111 posts

Re: Deciphering language processing in the human brain through LLM representations

#71

Post by pessimizer »

vkazanov wrote: post #63
famouswaffles wrote: post #43

Earlier quoted context omitted.

>They mention a profound difference in the opening paragraph, "Large language models do not depend on symbolic parts of speech or syntactic rules. "Human language models very obviously and evidently do. Honestly do they ? To me, they clearly don't. Grammar is not how language works. It's useful fiction. Language even in humans seems to be a very statistical process.

Yes! As somebody who speaks 2 languages, and sort of reads/understands 2 more, I cannot agree more. Human spoken languages do not follow any grammars. Grammars are just simplified representations of reality that is probabilistic in nature. This is something that Chomsky got very wrong, and the statistical/ML crowd got very right. But still, grammars are a very useful model.

Languages definitely follow grammars. They don't follow the grammars that were written by observing them, but you can discover unwritten grammatical structures that are nevertheless followed by everyone who speaks a language, and who if asked wouldn't even be able to articulate the rules that they are following. It's the following that defines the grammar, not the articulation of the rules.

Statistics are just another way to record a grammar, all the way down to the detail of how one talks about bicycles, or the Dirty War in Argentina.

If a grammar is defined as a book that enumerates the rules of a language, then of course language doesn't require following a grammar. If a grammar is defined as a set of rules for communicating reasonably well with another person who knows those same rules, then language follows grammars.

Re: Deciphering language processing in the human brain through LLM representations

#72

Post by probably_wrong »

My mildly grumpy opinion: this is not the first paper to show correlation between brain activity and the layers of a transformer. I know that Wang et. al (2024) have done it last year[1], but I doubt they're the only ones - I just have them in my head because I was reading their paper last week. Bonus fact: Wang et. al's paper also shows that test scores are a relevant factor in said correlation.

The point that always comes to mind is: correlation does not imply causation. I guess the main contribution would be a better mapping of the areas of the brain associated with speech production, but jumping from "these two things correlate" to "these two things are essentially the same" seems to me a bit of a stretch.

[1] https://arxiv.org/pdf/2407.10376

Re: Deciphering language processing in the human brain through LLM representations

#73

Post by vkazanov »

pessimizer wrote: post #71
vkazanov wrote: post #63

Earlier quoted context omitted.

Yes! As somebody who speaks 2 languages, and sort of reads/understands 2 more, I cannot agree more. Human spoken languages do not follow any grammars. Grammars are just simplified representations of reality that is probabilistic in nature. This is something that Chomsky got very wrong, and the statistical/ML crowd got very right. But still, grammars are a very useful model.

Languages definitely follow grammars. They don't follow the grammars that were written by observing them, but you can discover unwritten grammatical structures that are nevertheless followed by everyone who speaks a language, and who if asked wouldn't even be able to articulate the rules that they are following. It's the following that defines the grammar, not the articulation of the rules. Statistics are just anothe…

> Languages definitely follow grammars

But it's the other way around! Grammars follow languages. Or, more precisely, grammars are (very lossy) language models.

They describe typical expectations of an average language speaker. Grammars try to provide a generalized system describing an average case.

I prefer to think of languages as a set of typical idioms used by most language users. A given grammar is an attempt to catch similarities between idioms within the set and turn 'em into a formal description.

A grammar might help with studying a language, and speed up the process of internalizing idioms, but the final learning stage is a set of things students use in certain situations aka idioms. And that's it.

> Statistics are just another way to record a grammar

I almost agree.

But it should be "record a language". These are two approaches to the problem of modeling human languages.

Grammars are an OK model. Statistical models are less useful to us humans but given the right amount of compute they do show much better (see LLMs).

Re: Deciphering language processing in the human brain through LLM representations

#74

Post by canjobear »

vkazanov wrote: post #73
pessimizer wrote: post #71

Earlier quoted context omitted.

Languages definitely follow grammars. They don't follow the grammars that were written by observing them, but you can discover unwritten grammatical structures that are nevertheless followed by everyone who speaks a language, and who if asked wouldn't even be able to articulate the rules that they are following. It's the following that defines the grammar, not the articulation of the rules. Statistics are just anothe…

> Languages definitely follow grammars But it's the other way around! Grammars follow languages. Or, more precisely, grammars are (very lossy) language models. They describe typical expectations of an average language speaker. Grammars try to provide a generalized system describing an average case. I prefer to think of languages as a set of typical idioms used by most language users. A given grammar is an attempt to…

This is a terminological difference. Linguists use "grammar" as a technical term for a speaker's implicit knowledge of how their language works. That knowledge could be statistical or rule-based in nature, although most linguistic theories say that it's rule-based. You're using grammars to mean human-produced descriptions of that knowledge.

Re: Deciphering language processing in the human brain through LLM representations

#75

Post by bondarchuk »

namaria wrote: post #34

This is interesting. The blog post links several papers, and I recommend reading them. Responses here however seem not commensurate with the evidence presented. Two of the papers[0][1] that provide the sources for the illustration in the blog post are about research conducted on a very small group of subjects. They measure neural activity when listening to a 30 minutes podcast (5000 words). Participants tried to gues…

>"Unlike humans, DLMs (deep language models) cannot think, understand or generate new meaningful ideas by integrating prior knowledge. They simply echo the statistics of their input"

[Citation needed]. Actually, the paper does give a citation (G.F. Marcus, the Algebraic Mind), that is from 2019 according to their citation list (i.e. before gpt3), but actually seems to be from the early 2000s.

Re: Deciphering language processing in the human brain through LLM representations

#76

Post by foobarqux »

vkazanov wrote: post #73
pessimizer wrote: post #71

Earlier quoted context omitted.

Languages definitely follow grammars. They don't follow the grammars that were written by observing them, but you can discover unwritten grammatical structures that are nevertheless followed by everyone who speaks a language, and who if asked wouldn't even be able to articulate the rules that they are following. It's the following that defines the grammar, not the articulation of the rules. Statistics are just anothe…

> Languages definitely follow grammars But it's the other way around! Grammars follow languages. Or, more precisely, grammars are (very lossy) language models. They describe typical expectations of an average language speaker. Grammars try to provide a generalized system describing an average case. I prefer to think of languages as a set of typical idioms used by most language users. A given grammar is an attempt to…

This is just wrong. Languages follow certain inviolable rules, most notably, hierarchical structure dependence. There are experiments (Moro, the subject "Chris") that show that humans don't process synthetic languages that violate these rules the same as synthetic languages that do (specifically it takes them longer to process and they use non-language parts of the brain to do so).

Re: Deciphering language processing in the human brain through LLM representations

#77

Post by Ey7NFZ3P0nzAe »

Ey7NFZ3P0nzAe wrote: post #70

I need you guys help. Is there some theorem stating something like random few-hot vectors can always be combined linearly to match any signal with a low p-value? I thought I encountered it sometimes in my experiments and that this might be happening in this llm x neuroscience trend of matching llm internals to brain signals.

Reply to myself to add more context as I'm at work: what I mean is that I suspect that the average "shape" of vectors in the llm is dependent on the depth inside the llm. In effect: vectors initially are gaussian-random but as the model gets trained they end up closer and closer to few-shot vectors, that are in effect easier to correlate to any signal including brain signals.

In effect: any kind of llm activation could be correlated to brain signals even though it's just a sophisticated p mapping and would not correspond to anything useful.

Iirc the team at meta ai of jean remi king shown that even randomly trained LLMs could be fitted.

Re: Deciphering language processing in the human brain through LLM representations

#78

Post by AgentMatt »

tkot wrote: post #55
achierius wrote: post #50

Earlier quoted context omitted.

Linguists however know that grammar is, indeed, important for linguistic comprehension. For example, the German "Ich sehe die Frau mit dem Fernglas" (I see the woman with the binoculars) is _unambiguous_ because "die Frau" and "mit dem Fernglas" match in both gender and case. If this weren't the case, it could be either "I see (the woman with the binoculars)" or "I see (the woman) with [using] the binoculars". Even i…

> For example, the German "Ich sehe die Frau mit dem Fernglas" (I see the woman with the binoculars) is _unambiguous_ because "die Frau" and "mit dem Fernglas" match in both gender and case. If this weren't the case, it could be either "I see (the woman with the binoculars)" or "I see (the woman) with [using] the binoculars". My German is pretty rusty, why exactly is it unambiguous? I don't see how changing the noun…

It is indeed ambiguous. I don't understand which alternative the parent is implying.

Re: Deciphering language processing in the human brain through LLM representations

#79

Post by vkazanov »

foobarqux wrote: post #76
vkazanov wrote: post #73

Earlier quoted context omitted.

> Languages definitely follow grammars But it's the other way around! Grammars follow languages. Or, more precisely, grammars are (very lossy) language models. They describe typical expectations of an average language speaker. Grammars try to provide a generalized system describing an average case. I prefer to think of languages as a set of typical idioms used by most language users. A given grammar is an attempt to…

This is just wrong. Languages follow certain inviolable rules, most notably, hierarchical structure dependence. There are experiments (Moro, the subject "Chris") that show that humans don't process synthetic languages that violate these rules the same as synthetic languages that do (specifically it takes them longer to process and they use non-language parts of the brain to do so).

What exactly is wrong? The fact that grammars are very limited models of human languages? My key thesis is that human languages operate in a way that non-probabilistic models (i.e. grammars) can only describe it in a very lossy way.

Sure, LLMs are also lossy but also much more scalable.

I've spent quite a lot of time with 90s/2000s papers on the topic, and I don't remember any model useful in generating human language better than "stohastic parrots" do.

Re: Deciphering language processing in the human brain through LLM representations

#80

Post by seanw444 »

BoiledCabbage wrote: post #57
seanw444 wrote: post #26

Earlier quoted context omitted.

A copy of you, not the same instance.

Do you create a new instance every time you awaken from sleep? If not, why not?

Because the same you has been active the whole time. Your brain didn't cease function through sleep. Kind of like sleep mode vs. rebooting a computer.
Post reply on HN