Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How the Brain Parses Language

quantamagazine.org

Post reply on HN
Print view View original

41–50 of 90 posts

Re: How the Brain Parses Language

#41

Post by rdtsc »

HarHarVeryFunny wrote: post #36
rdtsc wrote: post #26

> But what if our neurobiological reality includes a system that behaves something like an LLM? With every technological breakthrough we always posit that the brain has to work like the newly discovered thing. At various times brains were hydraulic, mechanical, electrical, like a computer, like a network. Now, of course, the brain has to be like an LLM.

Yes, but at least now we're comparing artificial to real neural networks, so the way it works at least has a chance of being similar. I do think that a transformer, a somewhat generic hierarchical/parallel predictive architecture, learning from prediction failure, has to be at least somewhat similar to how we learn language, as opposed to a specialized Chompyskan "language organ". The main difference is perhaps that…

> Yes, but at least now we're comparing artificial to real neural networks, so the way it works at least has a chance of being similar.

Indeed, and I wasn't even saying it's wrong, it may be pretty close.

> What would be cool would be if we could to some "mechanistic interpretability" work on the brain's language generation circuits, and perhaps discover something similar to induction heads.

Yeah, I wouldn't be surprised. And maybe the more we find out about the brain, it could lead to some new insights about how to improve AI. So we'd sort of converge from both sides.

Re: How the Brain Parses Language

#42

Post by adamzwasserman »

lapcat wrote: post #35
adamzwasserman wrote: post #29

Earlier quoted context omitted.

The body of knowledge on Ericksonian hypnotherapy is pretty clear that the effect of language on Level 1 is orthogonal to, and sometimes even opposed to, conscious processes. I became interested after being medically hypnotized for kidney stone pain. As the hypnotist spoke, I was consciously thinking: "this is dumb, it will never work." And yet it did. That's exactly your point — I was fully conscious and "at home" t…

I think you're creating a false dichotomy between meta-thinking and mere reflex, when in fact most conscious thinking is neither of those. My understanding is that a hypnotized person is very focused on the hypnotist and suggestible but can otherwise carry on a relatively normal conversation with the hypnotist. And certainly an unhypnotized chattering person is still conscious, aware of the context as well as the sub…

I don't think we're far apart. My claim isn't that Level 1 is "mere reflex". It's that language can produce effects at a level that operates independently of (and sometimes in opposition to) conscious evaluation. The hypnosis example is just a clean demonstration of that separation.

Whether LLMs are useful models for studying that level is an empirical question. They're not conscious, but they do learn statistical regularities in language structure, which may be exactly what Level 1 is optimized for.

Re: How the Brain Parses Language

#43

Post by adamzwasserman »

Grosvenor wrote: post #39
adamzwasserman wrote: post #19

There's an interesting falsifiable prediction lurking here. If the language network is essentially a parser/decoder that exploits statistical regularities in language structure, then languages with richer morphological marking (more redundant grammatical signals) should be "easier" to parse — the structure is more explicitly marked in the signal itself. French has obligatory subject-verb agreement, gender marking on…

And we have those French/English text corpora in the form of Canadian law. All laws in Canada at the federal level are written in English and French. This was used to build the first modern language translation systems, testing them going from English->french->english. And in reverse. You could do similar here , understanding that your language is quite stilted legalese. Edit: there might be other countries with simi…

Incredibly, I had not thought to use that data set.

Now I will. Thanks.

Re: How the Brain Parses Language

#44

Post by jimbokun »

rdtsc wrote: post #26

> But what if our neurobiological reality includes a system that behaves something like an LLM? With every technological breakthrough we always posit that the brain has to work like the newly discovered thing. At various times brains were hydraulic, mechanical, electrical, like a computer, like a network. Now, of course, the brain has to be like an LLM.

All of those analogies were useful in some ways, and LLMs are too.

There's also a progression in your sequence. There were rudimentary mechanical calculating devices, then electrical devices begat electrical computers, and LLMs are a particular program running on a computer. So in a way the analogies are becoming more refined as we develop systems more and more capable of mimicking human capabilities.

Re: How the Brain Parses Language

#45

Post by jimbokun »

Timwi wrote: post #10
Al-Khwarizmi wrote: post #4

Earlier quoted context omitted.

Ev Fedorenko is a highly recognized cognitive scientist that has been studying how humans parse language for years. Of course this doesn't mean one shouldn't question what she says (that would be an obvious authority fallacy), but I do think it's fair to say that if you want to question it, the argument should be more elaborate that "this sounds like she has no idea of the topic".

I'm not the person you responded to, but I found the article unreadable because it kept going on about Ev’s life instead of her research. I'm sure her research is valuable and insightful, but with this style of reporting it is both inaccessible to me, and it gives me the (probably flawed) impression that her research isn't the part of her life that's supposed to be important or impressive.

This is meant for a lay audience so you should probably just read her research papers.

Also:

> it gives me the (probably flawed) impression that her research isn't the part of her life that's supposed to be important or impressive.

I don't see this at all in the article. There's just some human interest content to make her research more approachable.

Re: How the Brain Parses Language

#46

Post by lapcat »

adamzwasserman wrote: post #42
lapcat wrote: post #35

Earlier quoted context omitted.

I think you're creating a false dichotomy between meta-thinking and mere reflex, when in fact most conscious thinking is neither of those. My understanding is that a hypnotized person is very focused on the hypnotist and suggestible but can otherwise carry on a relatively normal conversation with the hypnotist. And certainly an unhypnotized chattering person is still conscious, aware of the context as well as the sub…

I don't think we're far apart. My claim isn't that Level 1 is "mere reflex". It's that language can produce effects at a level that operates independently of (and sometimes in opposition to) conscious evaluation. The hypnosis example is just a clean demonstration of that separation. Whether LLMs are useful models for studying that level is an empirical question. They're not conscious, but they do learn statistical re…

> language can produce effects at a level that operates independently of (and sometimes in opposition to) conscious evaluation

I don't think this is a particularly interesting claim if "conscious evaluation" is understood so strictly that it excludes an ordinary motormouth.

Re: How the Brain Parses Language

#47

Post by taeric »

This feels too reductive to me. In particular, it makes a hard distinction between the thinking and the language. I fully accept that they are distinct, but how distinct? It is hard not to think that some thinking styles influence how something is heard?

Not just in full language, mind, but consider the last time you heard a song in a major key? Do you even know what that means? Because many of us do not.

Same goes for listening to people discuss things like sports. I'm inclined to think many people effectively run a simulation in their mind of a game as they listen to it broadcast. This almost certainly isn't inherent to the language, it is part of the learning of it, though. Think looking over lists of the moves in a chess game. Then go from that to laying out the pieces as they are after that list. Or calling what the next move can be.

Can this be a completely separate set of "circuitry" in our brains that first parses the language and then builds the simulation? I suppose. Seems more likely there is something that is active between the two that can effectively get merged in advanced practitioners.

Re: How the Brain Parses Language

#48

Post by mzs »

alfanick wrote: post #7

Anecdotal data, based on a sample of 1 (aka me). I'm originally Polish, but I would say my mother tongue is English. I also learned Latin as a kid/teen. Then learning any other languages is much easier, I also learned German and some Swiss German dialects. I can also do Spanish, Italian, French, Dutch, Czech, some Serbo-Croation. I think being Polish makes learning languages easy - as we have a lot of creations in Po…

I completely understand! I'm also Polish American. I have to say it helps when mother's side of family is Gdańsk+west and father's Lublin+east. My wife's family is all from Warsaw area and I had to translate for my father-in-law during a holiday to Władysławowo-Hel (probably helps my aunt's father's side is Kashubian too, mmm... dessert first).

I was blown-away on holiday to Croatia. It was so unexpectedly relatively easily understandable after Czechia, Austria, and Slovenia. I was all, "What just happened!? Shouldn't this be something more like Italian?"

It took only a month for me to be able to communicate in Ukrainian with my ESL students, you're totally right about Cyrillic. And I too think in concepts but switch my brain to express them externally via language, whatever that language may be at the moment. I am terrible at translating OTOH, so unnatural!

But it has it's limits, I got to a point after German and Norwegian that I thought I harbored a super-power. Then I went to school in Hungary ;) I also had an ESL student from Lithuania, yep incomprehensible.

Re: How the Brain Parses Language

#49

Post by lukeinator42 »

liampulles wrote: post #9

What I'm curious about is what the language parts of the human brain look like for babies and toddlers. Humans obviously have a bunch of languages they can speak, and toddlers pick up the language that their guardians speak around their home, so there seems to be machinery there that is for the task of "online" learning.

It's an interesting area of research, there is even some evidence that language experienced in utero affects speech perception: https://doi.org/10.1111/apa.12098.

Re: How the Brain Parses Language

#50

Post by tgv »

adamzwasserman wrote: post #19

There's an interesting falsifiable prediction lurking here. If the language network is essentially a parser/decoder that exploits statistical regularities in language structure, then languages with richer morphological marking (more redundant grammatical signals) should be "easier" to parse — the structure is more explicitly marked in the signal itself. French has obligatory subject-verb agreement, gender marking on…

There are more differences between English and French than you just described, and they can affect your measurement. Even the corpora you use cannot be the same. There isn't "ceteris paribus" (holding everything else constant). The outcome of the experiment doesn't say anything about the hypothesis.

You're also going to use an artificial neural network to make claims about the human brain? That distance is too large to bridge with a few assumptions.

BTW, nobody believes our language faculties are doing the thinking. There are however, obviously, connections to thought: not only the concepts/meaning, but possibly sharing neural structures, such as the feedback mechanism that allows us to monitor ourselves.

I have a slightly better proposal: if you want to see the effect of gender, genderize English or neutralize French, and compare both versions of the same language. Careful with tokenization, though.

Post reply on HN